How fast is FileMaker Server 19.1.2?

Find operations on FileMaker Server 19.1.2 are up to 10x faster than before

Claris released a new version of FileMaker Server with features that improve the performance and stability of your hosted FileMaker apps. My favorite addition is “parallel indexed finds,” which means that users can now run multiple find operations on a field simultaneously, unlike previous versions. So how fast is FileMaker Server 19.1.2?

Because a sharing lock is used to allow concurrent read access, version 19.1.2 can now perform find operations on indexed fields in parallel. Previously, only one indexed find operation could be performed at a time. 

You can read more about these changes in the Claris articles Performance, stability, and reliability improvements in Claris FileMaker Server 19.1.2 and in the FileMaker Server 19.1.2 release notes.

This post focuses on testing parallel indexed finds. Wim Decorte created a tool known as Punisher, which can be used to analyze the impacts of different environments on FileMaker Server.

The goal of my tests was to see if there is a noticeable performance improvement in version 19.1.2. Because I didn’t find significant performance differences between FileMaker Server version 18.0.4 and 19.0.1, I skipped version 19.0.1 in my comparison. Moreover, I did not use the startup restoration feature introduced in version 18 due to its negative impact on the server’s reliability and stability. None of our cloud servers use this feature, so I omitted it from the comparison.

Test Objective:

• Simulate an environment with multiple users running find operations at the same time
• Check for performance differences between FileMaker Server 18.0.4 and FileMaker Server 19.1.2 when executing multiple expensive find operations

Speed Test Methodology:

• Install and configure FileMaker Server 18.0.4.428 (Startup restoration OFF) and 19.1.2.234 (Sharing Lock ON) on 2 separate AWS t3.medium instances with the following specifications:
  • Windows Server 2019
  • 2 vCPUs
  • 4 GB Memory
  • 10 GB gp2 (SSD) volumes for the FileMaker databases
  • 50 GB gp2 (SSD) OS Volume
  • Latency from my computer to the server: 18 ms
• Run the following test 12 times per server:
  • Use a FileMaker database with 2 million fake person data records in one table
  • Run 19 server scripts (PSoS) simultaneously to simulate other users using the FileMaker database at the same time
  • Each server script (PSoS) runs 4 random and expensive find operations
    • The find keyword is a string with 3 letters separated by spaces (e.g. “a b c”)
    • These are among of the worst queries that can be done on a single table of this size without related tables; they make FileMaker server work really hard
  • While the PSoS scripts are running, run queries on local FileMaker Pro client

Notes: Filemaker Pro 18.0.3.317 is used with FileMaker Server 18.0.4.428 (Startup restoration: OFF). FileMaker Pro 19.1.3.315 with FileMaker Server 19.1.2.234 (Sharing lock: ON). The target field is “full_name”, a fully indexed calculation field for First Name + Middle Name + Last Name.

Speed Test Results:

FileMaker Server 18.0.4.428:

Query performance for a single user:
MIN: 600 ms; AVG: 1,164 ms; MAX: 1,749 ms

Query performance under load (20 user sessions):
MIN: 15,290 ms; AVG: 168,148 ms; MAX: 218,282 ms

FileMaker Server 19.1.2.234:

Query performance for a single user:
MIN: 312 ms; AVG: 1137 ms; MAX: 1751 ms

Query performance under load (20 user sessions):
MIN: 1,082 ms; AVG: 13,252 ms; MAX: 22,894 ms

It’s worth noting that the server CPU was at 100% while these test runs were running, which means that response times for other server operations were slower.

One Caveat

The average expensive query time in the outlined test is more than 10 times faster in 19.1.2 than in 18.0.4. However, it’s important to note that the test method utilized runs expensive queries. These are queries that take more than 1000 ms to finish executing and utilize 20% or more of the server’s CPU.

I found that lightweight queries (taking between 1 and 70 ms) are not as affected by the described find operation improvements in 19.1.2 in a 20 user environment. In fact, I found that the performance was almost exactly the same on 18 and 19.1.2 for lightweight queries. The lightweight queries included in testing searched a list of 50 first names in the full name field among 2 million person records while 20 other server scripts were running the same 50 queries on the same field. Theoretically, you should notice performance differences in these queries in larger environments with 50 or 100+ concurrent users.

FileMaker Speed Test Summary & Conclusion

There are performance benefits in FileMaker Server 19.1.2 over 18.0.4 in a specific scenario – running expensive queries on a server busy with other expensive queries on the same field. I found query speed to be more than 10x faster in FileMaker Server version 19.1.2 over 18.0.4 (or 19.0).

Expensive queries utilize a large amount of the server CPU. If expensive queries can run faster, the server is more available for other operations. Parallel indexed finds therefore have an indirect positive impact on non-query related server operations, in addition to better find performance.

Let us know in the comments what you think about these results and whether you found other changes in 19.1.2 that impact your FileMaker applications.