Converting databases from previous versions

Databases created with previous versions of 4D are compatible with 4D v13 (structure and data files).

• Version 11 or 12 database files are converted directly to version 13. Once converted, the structure file can no longer be opened in version 12; however, the data file can be opened in version 12 again under certain conditions described below.
• Database files in versions 6.x, 2003.x or 2004.x are converted through a wizard and can no longer be opened with their original version.

Note: You can convert any interpreted structure file. The file may contain compiled code; in this case, you will need to recompile the database after its conversion.

A database from version 11 or 12 gets converted directly when you open the structure file with 4D v13. Two successive warnings let you know that the files are being converted and that you can no longer open them with an earlier version.

Note that when you convert a data file, its indexes are rebuilt.

Note: If you want to convert a v11 database that uses an earlier version of 4D Pack, we reconmmend that you install the most recent version of the 4D Pack v11 plug-in in the original database before proceeding with the conversion.

The "Macros.xml" file lets you use macro commands in the Method editor (see Creating and using macros). It is located in the user preferences folder:

• Windows XP:
  C:\Documents and Settings\UserName\Application Data\4D\Macros v2\
• Windows Vista / Windows 7:
  C:\Users\UserName\AppData\Roaming\4D\Macros v2\
• Macintosh:
  {Disk}:Users:UserName:Library:Preferences:4D:Macros v2:

In version 12 of 4D, there are new macro commands available. Since the "Macros.xml" file may have been customized, when you install a new version 4D does not automatically replace the existing version of this file.

To use the new SQL macro commands of 4D v12, you must either:

• delete the "Macros.xml" file in the "Macros v2" folder (if you have never modified it), then launch 4D to create the new file automatically.
• add the new macros manually to the "Macros.xml" file in the "Macros v2" folder (if you have already customized the contents). The new template file for the macros is located in the 4D application folder 4D\Resources\en.lproj or 4D\Resources\fr.lproj.

4D v13 can open v12 or v11 components (compiled or interpreted) directly without conversion or a confirmation dialog box. Remember that components are always open in read-only mode.

You do not need to recompile components but conversion to v13 is only possible for .4DB files and not for .4DC ones.

The table below presents the retro-compatibility scenarios for the files of 4D databases that are converted into v13, i.e. the possibility of opening them again in previous versions:

(*) You can still open the converted data file in version 12 by explicitly allowing this on the version 12 side. This is done using the Allow opening v13 data files preference found on the “Database/Data Management” page of the Database Settings:

By default, the No option is selected. You can choose Yes to allow data to be opened directly or Ask in order to display a confirmation dialog before opening the data. 

This option is intended for recovering data in specific cases and should be used with precaution:

• Before reopening the data: in all cases, we recommend deleting the v13 index file (.4dindx file).
• After reopening the data: when new functions of version 13 have been applied to database tables, we recommend compacting or repairing the data file using the Maintenance and security center.

Structural changes made at the 4D database engine level require in-depth conversion of both the structure and data of earlier databases using a specific wizard when converting to versions 11 and 12. As a precaution, this wizard makes a copy of the original database before converting it so you can revert to it any time.

You can convert any interpreted structure file. The file may contain compiled code; in this case, it will be necessary to recompile the database after conversion.
We recommend that you check the integrity of the database using the 4D Tools utility (compacting, verifying and, if necessary, repairing the database) before conversion. Otherwise, if an anomaly is detected during conversion, the procedure will be stopped and the conversion wizard will ask you to use 4D Tools. Use the 4D Tools version that corresponds to your original database.

To convert a previous database, simply select it in the opening dialog box of 4D (see Opening a local database). The conversion wizard appears automatically: 

Click on the Convert database button to start the standard process for converting the structure and data files. You can also display or modify the default parameters used for conversion by clicking on the Details > button (see the following paragraph).
As a precaution, the conversion wizard will systematically make a complete copy of the database (structure and data) before beginning the procedure. If the conversion fails, you will therefore still be able to recover your original database intact. Duplicating the database requires additional disk space. If the disk space available is insufficient, a warning message will appear and the conversion will be stopped. 

The original files are copied into a folder named Replaced Files (Conversion), which is created next to the original files. If the data file is in the same folder as the structure file, the Replaced Files (Conversion) folder will contain both original files. If the data file is in another folder or on another volume, the conversion will create a Replaced Files (Conversion) for each file. 

Note: The current log file of the converted database is also copied into the Replaced Files (Conversion) folder and a new blank log file is created. 

If the wizard encountered any non-critical errors during conversion, they will be recorded in a conversion log file named DataConversion_Log.log that will be placed next to the converted structure.

Any older or obsolete mechanisms that are no longer supported are removed or replaced during conversion. For more information, refer to the 4D v11 SQL Upgrade (PDF). For a detailed description of all the modifications, refer to the Conversion to 4D v11 SQL (PDF).

If you want to view or change certain parameters before carrying out the conversion, click on the Details > button. The dialog box then displays various information concerning the conversion parameters. You can open the different pages of information by clicking on the corresponding areas:

• Structure File Information: This page shows the size and location of the original database structure file as well as the future location of the precautionary copy that will be made of this file.
• Data File Information: This page shows the size and location of the original database data file as well as the future location of the precautionary copy that will be made of this file. It can also be used to specify the data file to be converted. By default, the current data file is selected. You can either create a new blank data file (Create a new data file option) or choose to convert another data file by clicking on the Choose another data file... button.
  For more information about converting several data files, refer to the paragraph "“Conversion of databases using several data files” below.
  The Change the destination... button lets you modify the location of the new data file after conversion.
• Segment Information: This page lists the segments of the data file associated with the database. For more information about this, refer to the “Conversion of databases with multiple segments” paragraph below. 
• Disk Information: This page indicates the space available on the disk of the database. 
• Option: This page contains a conversion option.
  
  • Do not execute code when opening database: When this option is checked, any code normally executed on database startup is deactivated when the converted database is launched for the first time. This option applies to code that is called using the On Startup database method. This allows you to avoid initialization errors that may occur following conversions.

After conversion, in addition to the .4DIndy and .4DIndx index files, 4D databases will contain the following additional elements:

• DataConversion_Log.log: This file stores any anomalies encountered during database conversion.
• Replaced files folder: This folder contains the copies of the original database files (structure, data and log) before conversion. 
• Catalog.xml: This file contains the description of the converted database structure as well as the new internal identifiers UUIDs used by 4D in version 12 (see Changing the data file). This file is required when you want to carry out several conversions of the structure while keeping the same data file. This can happen in the case where an application is developed in version 2004 with an initial conversion to v12 followed by successive conversions of the structure file during the development phase, while the converted v12 application is in use. Only the structure file is therefore converted subsequently.
  Warning: Using the Catalog.xml file to perform multiple conversions is a tricky operation that entails the risk of data loss in the event of a handling error. We recommend that you do not allow users to perform this operation on their own without immediate data verification. Generally, this specific mechanism must be reserved for particular cases; it is not recommended to continue development in version 2004 once deployment has been carried out in v11 or v12.

If the database that you want to convert uses several different data files, the conversion of the database and each of the data files will be performed separately. Conversion of additional data files is carried out using the Open>Data File command of the File menu. 

1. Converting a database with its current data file.
2. In the converted structure, choose the Open>Data File... command in the File menu or in the menu associated with the “Open” button of the 4D tool bar.
   A standard Open document dialog box appears so that you can indicate the data file that you want to convert. 
3. Select the data file to be converted and click on Open.
   The conversion wizard window then appears. The information concerning the data file is updated depending on the file selected. The data file to be converted will first be duplicated and placed in the Replaced Files (Conversion) folder, according to standard conversion principles.
4. Validate the wizard window in order to launch the conversion.
   After the conversion is finished, the converted data file becomes the current data file of the database.

It is not possible to convert a structure file containing a previous generation component (prior to version 11). It is imperative to uninstall any components using the corresponding version of 4D Insider (utility available up to version 2004) before proceeding with the conversion of a structure file. 

For more information about “new generation” components, refer to Developing and installing 4D components.

Starting with 4D v12, the data file size is virtually unlimited (except for the limits imposed by the operating system) so you can no longer create or use data segments. 

When you convert an older database that contains segments, the conversion wizard groups the contents of all the segments into a new data file. You must make sure that you have a sufficient amount of disk space for the new data file.

The segments of a database can be seen on the “Segment Information” page of the conversion dialog box (see the “View or change conversion parameters” paragraph above. 

If a segment is missing during conversion, the conversion wizard lets you indicate its location manually. If the segment cannot be located, conversion cannot take place.

When converting a database to version 12, textual data are converted into Unicode. In order for the conversion to be performed correctly, 4D must know the source character set. By default, the character set corresponding to the current system language is used.
In previous versions of 4D, it was possible to associate a specific character set to form objects via the Keyboard Layout property. In this case, 4D is not able to deduce the source character set with certainty since the property was stored with the form and data entry could be carried out via intermediary variables.

It is up to the developer to explicitly declare the specific character sets used before conversion, via a text file placed at the same level as the data file to be converted. This file must have the following characteristics:

• Name: multilang.txt
• Encoding: ANSI or Mac Roman (do not use Unicode)
• Format: table_number separator field_number separator subfield_number (optional) separator dialect_code CRLF
  
  • The separator is “;”
  • Each row must be ended with a carriage return (CR or CRLF), empty rows and spaces are allowed.

Example:
Given a database with 4 tables (TABLE_1 to TABLE_4).

• In TABLE_3, there are 4 Alpha fields (field_1 to field_4)
• In TABLE_4, there are 6 Longint fields and a SUBFIELD subtable with 2 Alpha fields
• The “Keyboard Layout” property (dialect) has been modified so as to be able to enter Greek in [TABLE_3 ]field_3 and [TABLE_4][SUBFIELD]field_1, and Russian in [TABLE_3 ]field_4 and [TABLE_4][SUBFIELD]field_2
• The multilang.txt file must therefore contain:
  3;3;1049
  3;4;1032
  4;7;1;1049
  4;7;2;1032
  The dialect codes are described in the keyboardmapping.xml file, located in the Resources subfolder.