Magento performance optimization

Magento is an excellent electronic commerce system, but it is server resource-hungry system and I used to see complaints on its low performance at the specialized forums.

Actually, after a small-scale study undertaken, we have found out that applying of the certain optimization will allow to achieve acceptable results. In this article I shall tell about necessary steps.

Server

Let's agree at once: we will proceed from the assumption that Magento  works on the dedicated server. If the server is virtual it is desirable to use XEN virtualization system. Open VZ has a number of limitations and often even very good optimized system works unacceptably slowly on this server. For example, the memory you see as accessible to you is not always real memory in RAM. It can sometimes be a swap, and it is no use talking about performance in such situation. Free memory in XEN is used to cache file system data, while file system data cache in Open VZ is common and neighbors compete for it if there is free memory for cache in node. And difference in the cost is insignificant. In general, all the recommendations will suit for VPS on Open VZ too, but if online store works slowly after all the optimization, change a hosting provider unhesitatingly.

Nginx

There are a lot of articles in the Internet on frontend-backend scheme subject, so there is no point in going through this theme. Some sources on this subject recommend to  install PHP-FPM as a backend. Perhaps, it gives advantages in a very loaded systems, but we have left Apache with mod_php as a backend, having used universal config for Nginx. The only condition is the website folder name must coincide with the domain name. In this case when adding a new website it is necessary to add website in Apache and there is no need to config Nginx correcting. The example of such a config is in the following article: «Universal config Nginx-Apache» (The article is in Russian, but configuration file is intelligible).

MySQL

Much depends on MySQL, server will not work quickly if its configure is incorrect. As description can be found in server documentation I intend to expand on basic parameter points.

innodb_flush_log_at_trx_commit = 2

tmp_table_size и max_heap_table_size - set value of each variable as 32-64 megabyte (values must be the same). These variables are responsible for the choice of the engine on which temporary tables will be created, any how we need Memory. The reason of slow reindexing is the creating of the temporary tables on the disk.

innodb_file_per_table=1 – this value does not influence on productivity directly, but this value is very convenient. When running the OPTIMIZE command, tables are recreated and occupy less place than before optimization. If however all data is stored in one file - it can expand. I will not describe the rest of the parameters, in the Internet one can find tones of information on server configure subject. In  general, the more base the more memory needs to be allocated for buffers. The task is to allocate memory that all the indexes and data are stored in buffers, as soon as memory is insufficient sudden loss of efficiency is evidenced. At the moment in our online-store  30000 goods have been downloaded, value  innodb_buffer_pool_size = 128M. As for now it is enough. By the way, after Percona Server using performance have increased in comparison with the original MySQL.

Opcode cache

We use APC, but I think analogues are possible to be used too. It is important not to use opcode cacher for Magento cache storing, because in this case it high fragmentation and works slowly. It is necessary to give cacher as much memory that it was enough for all scripts cache – 64   megabytes is enough, and of cause it must be stored in memory. If you use multisite mode it is nesessary to modify index.php file in Magento root directory, for  $compilerConfig and $mageFilename variables indicate absolute path for relevant files, otherwise cacher will cache php for each site separately.

Sessions storage

I read different recommendations and after experiments have come to the conclusion that best way is to store sessions in  Memcache. In this case there is no need to clear out of date sessions, and again: these sessions are storing in memory. There are some disadvantages: sessions appears to be lost when server reboots and sessions do not block (the second disadvantage is just theoretical, we did not find any errors). For now we put up with these disadvantages but if it is unacceptably it is possible to use Membase server. Membase server is full compatible with Memcached, but it provides persistence and list blocking, in this way you will have to wright  Session SaveHandler. The only – it needs minimum 100  megabyte of memory for work. By the way the function of  clean of out of date data which TTL have already is up of time is realized in Membase. 

Under no circumstances use sessions storage in MySQL! Besides the base growing it leads to the significant work delay.

Magento cache storage

Being  built on Zend Framework Magento cache subsystem support tags.  It is possible to reset  a part of cached data trough the tags, but not all the cache storage variants support tags. Lets review which variants are in Magento base:

 1.       Filesystem – not good! Works slowly but support tags.

2.       MySQL – not good too! Works slowly plus loads on database. Tags are supported too. 

3.       Opcode cacher – better not to touch, its task is cache opcode. Does not support tags.

4.       Memcached – does not support tags.

5.       TwoLevels cache - The gist is that data cached is going in a fast cache (for example in Memcached) and in a slow one – filesystem or MySQL. But if cache is under intensive using, in other words much data are stored there, problems with productivity will be. Besides, cache is reset by tags just in a slow cache and in a fast cache it must invalidate by itself. It is inconvenient as it is much better when reset cache is going instantaneously. In another way it is impossible to cache some things like “My cart” block, because goods adding to the cart must reset cache instantaneously.

 As it appeared there is a way out – there is such a project as  memcaсhe_tag, better say memcached_tags, exactly this project works steady (the conclusion is going from reviews and from our server testing).

 To tell the truth I was really surprised that this project is not described seriously and is not popular. Exactly this project has solved our problems. Everything is caching in the memory and with tags!  To switch on the memcaсhe_tags project support we had to develop a module, but the results exceeded our expectations. Say the truth, memcache_tags based on a rather old version of memcached, but it is functioning properly.

Memcached configuration

As a result, we have ascertained that in order for everything to work fast, we use Memcached to store sessions and cache. But to store sessions and cache in the one daemon is not exactly correct, so you need to run two memcached daemon on the same server but on different ports, and then register in the Magento configuration settings for saving sessions and cache in Memcached.  Instructions on multiple daemons on one server running are here: How can i run multiple instances of memcached on one server?

Compilation

Well, of cause a compilation must be enabled, in this case Magento will not search for the necessary php file in code pools at each request, in other words Magento will not sequential run “core”, “community”, “local”. Plus, all the main files are merged into a few large files. So enable compilation surely.

What is the result

As a result, rate of generation of the page the catalog or goods supposed to be 0.8-1 per second at an average VPS. If it is not so you need to watch the results of the profiler. If the results are jumping from time to time, often at random it means that hosting provider oversell in 90 percent of the 100. There may be a reasonable question - why should you organize all these manipulations with memcached if you can create a tmpfs directory for cache and sessions? First, when using tmpfs strange errors arose,  secondly, memcached automatically purge old or unused data, if we used tmpfs we would have to allocate more memory, otherwise the disk space can come to a close and then the system will fail. Third, this configuration is much more scalable. We can always make a separate Memcached (or Membase) server or to organize a cluster of servers. In other words, the system immediately becomes horizontally scalable. In the next article I will talk about our caching module. Its developing was possible by the successful solution of the cache storage problem through the use of memcached-tags. After installing it on the server response time to user action is from 1 second (there is nothing in the cache) to 0.3 seconds. In fact, with this variant, the user very comfortable working with.