Amasty Research | Full Page Cache vs. Varnish Cache

Table of Content

Amasty research: Full Page Cache vs Varnish Cache
Posted in: Magento 2 Guides & Tricks
by
33 comments

The Amasty Team has recently released a new extension — Full Page Cache — designed to speed up Magento store pages, which are often known for their heavy load. Full Page Cache is already proving to be a great success: it significantly reduces page load times and includes several other valuable features.

But actions speak louder than words, and statistics speak even louder.
Check out Amasty's latest research: How fast can Magento be with Full Page Cache? 

How Fast Can Magento Be With Full Page Cache?

TEST CONFIGURATION

HARDWARE

  • CPU: Intel Xeon L5520 2.27GHz (16 core)
  • RAM: 16 GB
  • Storage: SSD INTEL SSDSC2BB480G4

SOFTWARE

  • Magento Community 1.8.1.0
  • Magento Sample Data 1.2.0
  • Apache 2.2.22
  • PHP 5.4.4 (mod_php + Zend OpCache 7.0.3)
  • MySQL 5.5.35

For testing Apache JMeter with the following scenario was used:

A visitor performs the following actions:

  • Opens the website homepage

  • Searches for “shirt” using the search field

  • Filters the results by price ($0.00 — $99.99)

  • Further filters the results by color (white)

  • Only one product remains; the user opens its product page

  • Adds the product to the cart from the product page

  • Navigates to the category page: < >Electronics / Cameras / Digital Cameras

  • Reloads the same category page

  • Adds the Canon PowerShot A630 camera to the comparison list

  • Adds the Kodak EasyShare C530 camera to the comparison list

  • Opens the product comparison page

  • From the comparison page, adds the Canon PowerShot A630 to the cart

The test was conducted with 10, 20, 30, 40, and 50 simultaneous visitors under two conditions:

  1. With only Magento cache enabled

  2. With both Magento cache and Full Page Cache (FPC) enabled

Before each test run, the Magento cache was cleared. A preliminary test with 10 simultaneous visitors was executed to warm up the cache, but its results were not included in the final analysis. This step ensures consistency and avoids skewed results due to cache buildup during successive runs.

For completeness, we also tested a variant with Magento caching completely disabled.
Impatient readers can jump straight to the chart below, showing the results for the product page under 50 simultaneous visitors.

 

Full Page Cache - Magento research

To eliminate the impact of internet data transfer speed, the test was conducted in “Remote start” mode. In this setup, Apache JMeter was run in server mode on the machine hosting the tested Magento installation. The server received the test plan from the controlling client, executed it, and sent the results back. The client side of Apache JMeter, running on a desktop machine, managed only the server-side execution and displayed the results. This approach ensured that internet connection speed did not affect the test outcomes.

Each test was performed 10 times to accurately calculate the average values.
For example, with 50 simultaneous users, JMeter executed the test scenario 500 times in total.

The test scenario includes 12 server requests per user (excluding the loading of CSS, JS, and image resources). Therefore, during the test with 50 concurrent visitors, a total of 6,000 requests were sent to the server.

The results, measured in milliseconds, are presented in the table below:

Full Page Cache - Magento research

You may have noticed that the Category Page is opened twice during the test. This is because when a product is added to the cart, the status of dynamic blocks, whose content must be reloaded on the next request, changes. As a result, the first page load from the cache is slower than subsequent loads.

To illustrate the difference in loading speed (once the dynamic blocks have already been refreshed), the test returns to the Category Page a second time. On this second visit, the page is fully loaded from the cache.

Based on the table data, the following charts have been created to provide a clearer visual comparison of the server response times.

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

The final chart presents the average results across all tested categories. On average, using the Full Page Cache Magento extension increased speed by a factor of five. In some cases, the improvement was even greater.

But how does our extension compare to Varnish Cache and is it actually better?

Magento Varnish Cache vs. Full Page Cache: A Comparison Test

If you need to speed up your Magento store, just ‘faster’ is not enough to describe the performance of your website. A fair contest of two Magento cache extensions with detailed speed results is at your service.

Read More: How to Set Up Varnish in Magento

TEST CONFIGURATION

We slightly modified the test configuration compared to our previous Full Page Cache test.

< strong>Hardware

  • CPU: Intel Xeon L5520 2.27GHz (16 cores)

  • RAM: 16 GB

  • Storage: SSD INTEL SSDSC2BB480G4

Software

  • Magento Community Edition: 1.9.0.0

  • Magento Sample Data: 1.9.0.0

  • Apache: 2.2.22

  • <strong "="">Varnish: 3.0.2

  • <p "="">PHP: 5.4.4 (mod_php + Zend OpCache 7.0.3)

  • MySQL: 5.5.35

Varnish Cache is a web application accelerator, also known as a caching HTTP reverse proxy. It is typically installed in front of a server that communicates over HTTP.

Since Varnish is not natively designed to work directly with Magento, we used the Nexcess Turpentine module to integrate Magento with Varnish for this performance test. Following the module provider's recommendations, we added the following parameters to the Varnish startup options: -p esi_syntax=0x2 -p cli_buffer=16384. The resulting startup options were:

DAEMON_OPTS=”-a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret 
-s malloc,256m -p esi_syntax=0x2 -p cli_buffer=16384″

For testing, we used the list of 14 URLs of Category and Product pages: http://our-test-site/

  • http://our-test-site/women.html
  • http://our-test-site/women/new-arrivals.html
  • http://our-test-site/women/new-arrivals/tori-tank-460.html
  • http://our-test-site/women/new-arrivals/elizabeth-knit-top-484.html
  • http://our-test-site/women/new-arrivals/lafayette-convertible-dress.html
  • http://our-test-site/women/tops-blouses.html
  • http://our-test-site/women/tops-blouses/nolita-cami-484.html
  • http://our-test-site/women/tops-blouses/black-nolita-cami.html
  • http://our-test-site/women/pants-denim.html
  • http://our-test-site/women/pants-denim/dumbo-boyfriend-jean.html
  • http://our-test-site/women/dresses-skirts.html
  • http://our-test-site/women/dresses-skirts/essex-pencil-skirt-527.html
  • http://our-test-site/women/dresses-skirts/ludlow-sheath-dress.html

Before every test launch, the Magento cache was cleared and a single visit to all URLs was performed to “warm the cache up” before taking the test results. After the warm-up test for 10, 20, 30, 40, and 50 simultaneous visitors was launched.

Comparison test results (in ms) are in the table as follows:

 

To make the results easier to understand and clearly show the page load speeds delivered by each Magento extension, we’ve created several diagrams.

Full Page Cache vs Varnish Cache speed review

Full Page Cache vs Varnish Cache extensions

Full Page Cache vs Varnish Magento extensions

Magento Full Page Cache Varnish Cache comparison

Magento cache performance comparison

As you can see, Full Page Cache outperforms the combination of Varnish Cache and Turpentine by 2 to 4 times. The most noticeable difference occurs during a user’s first visit to the site when both caching solutions create a new session and initialize Magento. Since Full Page Cache operates at an earlier stage in the request lifecycle, it can deliver a cached result faster. In contrast, Turpentine spends additional time on unnecessary initialization. This is crucial, as first impressions significantly impact user experience.

According to Kissmetrics, even a 1-second delay in page load time can reduce conversions by up to 7%. You might want to consider the potential revenue loss caused by using a less efficient Magento caching solution.

That’s why the Amasty Team recommends using the Full Page Cache Magento extension to achieve the lowest possible page load times.

June 5, 2014
June 12, 2014
April 1, 2014
Related posts
Related products
Comments
Wes
June 6, 2014
Hi Andrew, Can you use FPC + Varnish + Turpentine together?
Reply
Vera Rabkina
June 6, 2014
Yes, sure.
Reply
Dave
June 18, 2014
No, https://github.com/nexcess/magento-turpentine/wiki/Compatibility
Reply
Vera Rabkina
June 18, 2014
Dave is right, my bad. Sorry for the wrong answer. FPC and Varnish can't be used together. Thank you, Dave.
Reply
Wes
June 19, 2014
Thanks Dave and Vera
Reply
Addison
September 3, 2014
Varnish means light speed. I used Lesti which provides about 62 ms with 100 visitors, 10000 requests. But with Varnish same benchmark this page loads in 6 ms. Varnish can process 2000 req/sec in my 4 core 3.4 GHz VM, 4 Gb RAM, SSD. I would prefer FPC because it is not creating trouble as Varnish. If you don't know VCL syntax your are blind. Also there are a lot of block that need to be set to no cache. Still get trouble with Community Pole. Nexcess says they solved this issue but I still have it in my 1.8 and 1.9 version out of the box, just testing. You vote and block remains unchanged. Vote again and again and nothing happening. If you go to other page you will see vote results. A new refresh will hide the block for that session (which is normal). Also there is a lack of support for Turpentine. Page Cache also provide a free version without ESI support (which is the most wanted). My conclusion Varnish is faster than any FPC on the market, the only problem is configuration, especially when you have AJAX in your theme.
Reply
Ksenia Dobreva
September 3, 2014
First of all, thanks for your comment! It is true that Varnish requires some work to be set properly so it gives max results, but that’s not exactly the point of the article. There are users that have time and knowledge to work on configuration, and there are users that need an extension out of the box. That’s why we did our tests like an average user would have done; we installed two products and compared their performance. However, it is great that you managed to get better results with Varnish.
Reply
Francesco Haymar d\'Ettory
February 22, 2018
Actually you worked on some systems configuration that was not out of the box, as switching Apache port, of course. I think you should underline mostly that your module does not need effort, while an additional server does need tuning and maintenance. An average user will not install it at all for this reason, while a system engineer will want to use it even with your module (yes, the incompatibility can be fixed) to improve performance even more.
Reply
Den
October 2, 2014
you dont understand the main difference between VARNISH FPC and other non-Varnish FPC. this is all about resources needed to process your requests. with Varnish you need 0 (zero) resources, with non-varnish fpc you will get to the limit very fast, more visitors/buyers you have - faster your server will die... speed is not so important as server capacity...
Reply
Ksenia Dobreva
October 2, 2014
Den, thanks for reading and commenting. I see your point, but in this article we tested the exact case, not the theory in general. We were interested in the results on a given website. Still, I can't agree on your comment about speed. Speed isn't less important than server capacity. It works for SEO in a great way (site speed is an important ranking factor), what is more, it directly influences customer behavior (here's just one of the hundreds of articles on website speed influence studies webperformancetoday.com/2014/04/09/web-page-speed-affect-conversions-infographic/) In other words, you can save money on your server, but lower site speed takes even more money from you. Cheers!
Reply
Rick Mangles
October 21, 2014
No, speed is not an IMPORTANT SEO FACTOR! That is a myth!!!! It's a factor...but not an important one.
Reply
Ksenia Dobreva
October 22, 2014
Hi Rick, thanks for your comment. We'd be grateful if you share some proof over that statement. If to add to the link shared above, correlation studies show that site speed is important. https://www.searchmetrics.com/knowledge-base/ranking-factors/
Reply
Rick Mangles
November 18, 2014
"Correlation does not imply causation."
Reply
Ksenia Dobreva
November 21, 2014
Indeed, Rick. But Google officially confirmed site speed as a ranking factor four years ago. https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html And from my experience it has become a stronger factor throughout 2014.
Reply
Den
October 2, 2014
I am sure that your test is invalid, the varnish response time is never dependent on the number of requests. 10 or 1000 requests - you will always get the same response time. and varnish even will be faster than any cache which is completely dependent on the database and php. Varnish is independent cache, directly from memory. even if you use ESI, there will be low impact. probably you misconfigured your test environment. i just tested many different non-varnish caching modules for magento, and they are awful, server is dying after 150-200 simultaneous users...Varnish keeps working.
Reply
Ksenia Dobreva
November 6, 2014
Hi Den, and thanks for your comment. Although, it looks like you're not taking into account the integration module and various ESI operating modes in Varnish. And in this particular case Varnish performance depended on Turpentine integration module. Hope it clears things a bit.
Reply
Den
November 6, 2014
LOL
Reply
Gershon
October 14, 2014
Hello, Want to share my experience: I have 3 sites on the same server 1) Use Amasty Full Page Cache musicgate.co.il (5000 products) 2) Varnish as fronted for static content and LETSI FPC (600 products) smoke4u.co.il 3) Varnish as fronted for static content and extendware FPC (1500 products) il-cosmetics.com Amasty Full Page Cache , work best out of the box. LESTI and Extendware with warnish get closed result.
Reply
Ksenia Dobreva
October 16, 2014
Hi Gershon, thanks for your comment and sharing your experience. Happy that you are using our extension and happy with the results.
Reply
Leon Poole
October 22, 2014
You should clarify in the article that you're testing against your Amasty FPC extension and not Magento Enterprise FPC native - this confused me while reading.
Reply
Ksenia Dobreva
October 22, 2014
Hi Leon, sorry if it confused you. In fact, we are working to publish a new article on this comparison soon.
Reply
Chris
April 24, 2015
Hi, after reading this article and some of the comments i think that many people will get confused because of the limited information given about the difficulties and the resources needed in each case. I will try to clarify my opinion by writing some facts combined with my personal experience and other developer experiences i have read. I don't consider myself an expert developer although i have worked in many projects during the last 20 years so before adapting my proposals please make your own research and benchmarks. Varnish can work on a separate server or even on many servers with load balancing, FPC solutions cant. Varnish operates in Virtual Memory where it stores all static content while FPC uses slow filesystem. If there is enough RAM and a page doesn't have any dynamic parts Varnish can serve it with no delay (2ms, just little slower than serving a static file which needs 0.7ms). Varnish is written in C and is massively multi-threaded while FPC depents on the webserver, PHP implementation and opcode cache. The amount of requests Varnish can handle on a load balanced multiserver environment where each dedicated server has lot of ram cant be compared to any other non custom solution i know. So i think i made it clear enough that in my opinion Varnish shine for websites with huge traffic allowing them to serve static content in a scalable ultra fast optimized way. But what about Magento dynamic parts, https, clearing cache and ram consumption? Well there is where pain starts, it is hard even for experienced developers using Varnish integration on Magento to configure non caching/lazy caching of custom dynamic parts, for example Turpentine vendor can't configure correct behavior of the default poll module even after trying for some time. There are no current plans to add HTTPS support to Varnish. Clearing cache can be a pain too, there are a bugs in Turpentine's integration considering cache refreshing and clearing and i am pretty convinced from both personal experience and countless posts that in all popular integrations cache will just keep growing until a manual delete (which will of course need time for cache recreation). As long as Varnish cache keeps growing the response time and RAM consumption will keep growing too so i would never use it on a server that hosts more websites and there is limited RAM. Personally my experience of using Varnish for websites that have a lot of dynamic parts is limited but people who manage large e shops claim that in this case Turpentine Varnish is slower than FPC, i am sure this is true because those developers are experienced and try hard for optimum results but i am also convinced that with a better Varnish integration this can change. Support is also another factor, with Varnish integrations support is needed a lot but because of the complexity there is none or limited while on paid FPC modules there is excellent one, although it wont be needed in most cases. So to summarize using Varnish needs root access on server so you can install it, exceptional skills to configure dynamic parts and correct integration bugs concerning cache refreshing and clearing and a lot of RAM. FPC on the other hand just need one click on Magento connect and you will have great results and one easy way to configure what custom parts will be cached and what wont be. For my needs a server with SSD, Apache with PHP-FPM, Zend opcode cache, Mysql, and FPC on Magento is fast enough but if i ever need to serve more requests i would use Redis for session management and cache backend. For even more requests i would find the bottleneck and depending on it and the costs of human and hardware resources i would consider taking advantage of Nginx with HHVM ( i made some tests and the results where really great) or more servers using Varnish.
Reply
Ksenia Dobreva
April 24, 2015
Hey Chris, thanks so much for your time and effort - you have shared great insights on this question! I'm sure our readers will benefit from your knowledge. Very impressive.
Reply
ADDISON74
July 30, 2015
Please check the latest version of Turpentine. The Poll problem is solved and many others. There is a new Turpentine team who is working great to improve this extension. Also Magento 2 will have support for Varnish 3 and 4 out of the box. I am using Turpentine latest + Varnish 3 + Magento 1.9.2 in production, one dedicated server, 16 RAM, SSD's, 8 cores, and I can say with a huge traffic daily this kick the ass. We tried before a lot of FPC's solutions and they are great for small websites with a less traffic. By the way Varnish will get https support officially, or you can use Pound in front of it which is easy to configure. I am working to find a way to get analyzing results correctly, because Varnish is hit first not the webserver (awstats takes info from apache logs). Nginx + HHVM could be a solution, it is very fast, or just HHVM doing the whole thing. In the last 2 years I configure many solutions, but always come back to basic because I encountered many issues. Even Lesti FPC has a problem with Poll side module (which is now solved in Turpentine), probably Amasty has it too.
Reply
Kenan SALTIK
June 2, 2015
As visitor number increases the response times of varnish decreases. What will If your concurrent visitor number will reach thousands....
Reply
Ksenia Dobreva
June 3, 2015
Hey Kenan, thanks for your comment! In case it's a 100% static page, performance will be defined by the web server and hardware. But if the page has dynamic elements, neither our FPC nor Varnish will be of great use in the case you described.
Reply
Francis Kim
November 9, 2015
Having fine tuned Magento instances to the max before, I'd say Varnish can be a bit overrated and its real life usage don't necessarily correlate to what some load tests may lead you to believe.
Reply
Ksenia Dobreva
November 9, 2015
Hey Francis, thanks for sharing your experience. So will you share some figures you got on real life projects?
Reply
Anabolika
January 2, 2016
Thanks Dave and Vera
Reply
Danish
August 31, 2016
hi can we use apc+memcached+redis / varnish+memcached+redis to speed up the performance.. or please suggest something to speed up the site. we have a marketplace with multi-seller.
Reply
Ksenia Dobreva
September 6, 2016
Hey there, normally we recommend Zend Opcache + Redis + Amasty FPC for the best performance. Cheers!
Reply
Thiago
October 14, 2017
You guys have the best cache extension. Hands down. But I keep having session and cart quantity, customer login problems. Any tips or specific documentation? Thanks and congratulations.
Reply
Alina Bragina
January 22, 2018
Hi Thiago, thanks for the great feedback! Pity you have the issue. Please, feel free to send a word to our support team at [email protected], they will be happy to see how they can help you with that. Thanks in advance!
Reply
Leave your comment

Your email address will not be published

This blog was created with Amasty Blog Pro

Join our blog newsletter!

Search

About author

Andrey Tataranovich

Andrei believes that only at work people can fully express themselves, so he does his best to implement all his knowledge while in the office. Having made computer science his hobby, Andrei is constantly developing his professional skills. But despite such a busy schedule he always finds time to read science fiction books and communicate with new interesting people.

Table of Contents

This blog was created with Amasty Blog Pro

Loading
© 2009-2025 Amasty. All Rights Reserved.