Using FileMaker’s Data Migration Tool

What is the FileMaker Data Migration Tool? Why would I want to use it?

To put it simply, the FileMaker Data Migration Tool, commonly referred to as the FM DMT, is a command-line tool that allows you to import data from one FileMaker file to another with one command line.

So why would I want to use it?

Claris’ documentation lists several reasons. It saves time compared to writing your own migration scripts. It performs faster as well. It maps fields for you based on ID and name, allows you to migrate user accounts, transfer custom value lists, and automatically adjust the next serial numbers.

But to me, all of these advantages boil down to one thing: the FM DMT makes having separated DEV and PROD environments a practical setup.

There are many reasons to have a separate DEV and PROD environment, but I think you and I both know the only one that matters: we don’t want our changes to break the whole production system. With a separate DEV environment, we can develop and test our app without worrying about breaking things in production.

Having two environments means we need to find a way to deploy new versions to production. This is where the FM DMT comes into play. I’ll demonstrate a standard deployment process you can adopt with the help of the FM DMT.

Create an account for using the FM DMT

Let’s start with preparation work. First, make sure you either have a full access account or an account with the extended privilege “fmmigration. “ This extended privilege is not there by default. So if you need to, create it under Manage Security.

Create a clone of the new version as the target file

Download the migration tool here. As of the writing of this article, the FM DMT is no longer locked behind the FileMaker Developer Subscription. It is now free for all FileMaker users. You do need to register for an account in the Claris Community to be able to access the download file, but I’m sure most of you have already done that.

There are three versions of the FM DMT: one for Windows, one for Mac, and one for Ubuntu (Linux). Make sure you get the one for the system you want to use for data migration.

Now, let me show you how to use the FM DMT for deployment. Please note that I’m showing the manual way of doing this to demonstrate the process. Nearly all activities I’m showing can be accomplished through shell-scripting, so feel free to use an automated procedure or build an FM app that leverages the Send Event script step to streamline the process.

First I want to get a copy of the new version I want to deploy. So, I’ll log into my DEV FMS, find the file I want to deploy, hit Close then Download.

If nothing happens after you click the Download button, check out this video detailing where to find your file ready for download.

Now I need to create a clone of my new file. To do this, I’ll open up the downloaded copy and use Save a Copy As… command to get a clone of my deployment file.

Alternatively, you can also use the backup feature in FMS to get a clone of the file directly.

Get a copy of your current version to be the source file

Now that I have my deployment file, I need to get a copy of the current version where all data is stored. Similar to what we did in step one, I will go to my PROD FMS admin console, close the file, and download a copy of it. Since it’s very similar to how we got the DEV file, I’m not going to show it here.

We want to keep the PROD file offline during the deployment to prevent any data loss during the deployment.

Use the FM DMT to migrate data from the source file to the target file

Now let’s use the DMT to migrate data from the old version to the new version. I prefer to move both the source file, all remote container data, and the target file into the same folder containing the DMT.

Now, open your command line tool. On Mac, it is Terminal; on Windows, it is the Command Line tool. Type in your migration command. I tend to prepare these commands ahead of time and test them out, so that I can simply copy and paste without having to write any code during deployment.

./FMDataMigration -src_path "FM_DMT_Demo_File_Old.fmp12" -src_account "DataMigration" -src_pwd "12345" -clone_path "FM_DMT_Demo_File Clone.fmp12" -clone_account "DataMigration" -clone_pwd "12345" -target_path "FM_DMT_Demo_File.fmp12" -ignore_valuelists

The syntax is very straightforward; you tell it which file is the source file and which file is the clone (target), provide credentials for both, and optionally give the new file a name. You can also use additional options to choose not to import certain data, like user accounts or value lists.

I usually disable importing value lists for deployment purposes because it’s common for new features to require updating value lists. So you want those changes to stay instead of getting overwritten by what was in the old version.

After data migration, I’ll read the migration log to make sure there are no outstanding issues. A migration log looks like this:

What I’m looking for are tables and fields that have NOT been migrated. Sometimes it’s normal to have tables and fields that aren’t migrated. For example, if I removed a table or a field in your new version, of course during migration the corresponding table and field in the old version won’t find a target and will fail to migrate.

However, if I spot anything that comes as a surprise, then I usually run the migration again, but this time under the verbose mode, by adding a “-v” to my command. The verbose mode will give me a much more detailed log down to the field level, which will help me explain why certain tables or fields did not migrate.

Upload the migrated file back to the PROD server

Then as the last step, I’ll upload the migrated file back to the PROD server.

If you have direct access to your server machine, then I’d recommend just copying and pasting your new file together with its remote container data into the Database folder, then opening the file via the FMS admin console.

Otherwise, you can use FileMaker Pro.

In this case, if you have external container data, make sure that you’ve already copied external container data to the same folder as the file you are about to upload. If you haven’t done this, the existing remote container data in the database folder will be wiped out after you upload the new file.

Depending on your solution, you may need to do some post-deployment checks.

Then that’s it. New file deployed. Pretty simple right?

Now that you’ve learned how to use the FM DMT for data migration, I’d highly recommend creating separate environments for your DEV and PROD activities. You can go with more than two environments if necessary; the process to promote a version from one environment to another will be very similar.