August 9, 2019

Test benchmarks for Rclone and NGINX to serve media files

Until recently we’ve been using Rclone serve for reading and writing files for the Stack. We decided to stress test our decision on one of our client’s WooCommerce site, and our benchmarks revealed slow loading times. That made us consider reading files using NGINX. So we put together some tests using Locust to see if serving files from NGINX would be a better option.

The results are pretty obvious. Although using Rclone to read and write files would have provided a consistent approach for the various cloud providers used to host a site with the Stack, we decided NGINX is mandatory.

The performance penalty for reads (which is roughly two orders of magnitude) comes from the way reads are performed — for each level of folders a file list operation precedes a read file. So in our case, this means Rclone is not fitting the use case.

We did experiment with the --vfs-read-chunk-size parameter. The results turned out better (slower by only one order of magnitude), but still not in line with our expectations.

So here are the slightly beautified numbers for readability purposes.

Test scenario 1 – 3 imagesTest scenario 1 – 3 images

We created a WordPress site with 3 images on the homepage:

  • 2 JPG images having 35 KB respectively 51 KB
  • a WEBP image weighing 468 KB

First we tested the load times for reading the files via Rclone. We generated up to 100 requests for the homepage and here are our findings:

Image Size
(KB)
Avg
(s)
P50
(s)
P90
(s)
P99
(s)
P100
(s)
image1.jpg352.11.682.993.794.56
image2.jpg512.612.893.274.094.26
image3.webp 4684.374.294.85.525.94

Avg represents the average loading time of the 100 requests. P50, P90, P99 and P100 represent the percentiles request times for each of the three images.

Now let’s have a look at the loading times for reading files via NGINX:

ImagesSize
(KB)
Avg
(s)
P50
(s)
P90
(s)
P99
(s)
P100
(s)
image1.jpg350.0440.0400.0430.1240.225
image2.jpg510.0430.0400.0440.1210.188
image3.webp 4680.1210.1180.1310.1970.221

Comparing the average loading time, NGINX is 30 to 60x faster, depending on image type and size.

Test scenario 2 – 14 imagesTest scenario 2 – 14 images

We ran a second test, this time with 14 images of different sizes for a more realistic scenario. We again generated up to 100 requests for the homepage containing these 14 images.

Here are the results when the readings are done with Rclone:

And here are the ones when the readings are done using NGINX:

Again, there is a considerable difference between readings done with Rclone vs NGINX, the latter being much faster. For example, if we take the average loading time of the two methods and compare them, NGINX readings are barely visible on the chart.

  • improvement
Go back