Any Magento 2 website well-populated with high-quality content and a whole bunch of products variety has to perform fast on the frontend. Why can’t we just say ‘should’ rather than ‘has to’?
It’s better to see once than to hear one hundred times:
Source: (based on HubSpot research telling how conversions depend on page load time).
Thus, can we say your frontend speed performance influences conversions? Sure, we can.
What are the top reasons your website can brake?
- Unminified JS and CSS files 68% (see the solution here)
- Slow HTML load speed 43%
- Uncompressed JS and CSS files 43%
- Uncached JS and CSS files 26%
To see a complete list of problems, use the source.
Optimize JS and CSS files, HTML code, and images with Google Page Speed Optimizer automatically. Increase your score in Google PageSpeed Insights right now!
What’s this got to do with indexing?
Indexing directly affects your storefront performance. Without indexers, we would force our store work on the fly. So, read on to learn what is indexing in Magento 2, why do you need it, how to create a new indexer and manage the process correctly.
- What is indexing in Magento 2?
- Magento 2: terminology of Indexing
- Why do you need Magento 2 custom indexers?
- How to create a Magento 2 custom indexer?
- Get the FREE 3-step guide with examples to create a custom indexer from the first try!
- Update On Save vs. Update By Schedule
- How to set up Cron job in Magento 2 for working with indexers?
- How to manage Magento 2 custom indexers through the command line?
What is indexing in Magento 2?
Put simply, indexing is the way Magento 2 converts refreshable data (products, categories, prices, etc.) to enhance the storefront performance. When you refresh the converted data it needs an update or Magento 2 to be reindex programmatically or manually.
More specifically, read the definition from Magento DevDocs:
Magento has a very sophisticated architecture that stores lots of merchant data (including catalog data, prices, users, stores, and so on) in many database tables. To optimize storefront performance, Magento accumulates data into special tables using indexers.
How does it look like in practice?
Let’s take an example from our Magento 2 Product Labels extension.
Say, you’ve created a product label and specified 4 conditions to determine cases when the label should be applied. Consequently, without indexers, while loading a product page, Magento 2 would have to check each label to verify it matches the conditions you preset. Definitely, this would take too much time, which could increase your abandoned cart rate and would decrease conversions in the foreseeable future.
In the case with indexing, information about what labels should be displayed on this or that product page is already saved in tables taking into account all the conditions.
By and large, if Magento 2 worked without these special tables with indexers, the performance would be slowed down and, as a result, you would lose a significant part of conversions, which is bad.
Now let’s dig deeper into the technical part.
Magento 2: terminology of Indexing
When laying it all out, indexing terminology comprises three main terms. They are:
- and Indexer.
What is dictionary?
The term includes all the original data entered into your system. In order to simplify the maintenance, these dictionaries are normalized, which means they are restructured to cut data redundancy and better data integrity.
What is index?
The term means the representation of the original data for improved reading/searching. They can include aggregations results, as well as different calculations. By using a special algorithm, an index content can be restored from a dictionary.
What is indexer?
Quite simply, the term implies an object creating an index.
When you understand why you need indexers and know the indexing terminology, it’s time for little coding. So, the next part we have dedicated to the creation of a Magento 2 custom indexer.
Why do you need Magento 2 custom indexers?
Getting back to Magento 2 Product Labels extension, when developing the module we came to the conclusion that we need to create a custom indexer for it.
What made us think so?
Just imagine multiple labels created on numerous conditions for a wide range of products.
Can Magento 2 make the analytical work (upon what label on what product pages to display) on the fly?
Yes, it can.
Will your storefront perform well at that.
We tested, it won’t.
That’s why we decided to create a special indexer and added a setting to the backend named ‘Use Indexes for Better Performance’, which allows you to enable the process of indexation:
Note that after the latest update the setting has been removed and now indexers are used by default.
How to create a Magento 2 custom indexer?
Get the FREE 3-step guide with examples to create a custom indexer from the first try!
No spam guarantee.
Update On Save vs. Update By Schedule
Update On Save mode
In the indexing mode, Update On Save has the sense to initiate the reindex for a definite instance after it’s been changed. In our case (with product labels) we depend on products. This means that while one concrete Magento 2 product is changed we need to reindex our instance for this product programmatically. For this, you can do the next:
- Create an app/code/Amasty/Example/etc/adminhtml/events.xml file:
- Create an app/code/Amasty/Example/Model/Observer/Backend/CatalogProductSaveAfterObserver.php file:
By doing so, we have added an observer for a product save action.
After a product is changed, the indexer will be called and the ID of the updated product will be transferred.
Update By Schedule mode
If you change the Update On Save mode for the Update By Schedule one, the app/code/Amasty/Example/etc/mview.xml config file will be read and MySQL triggers will be created. This means that while deleting, adding, updating rows in the defined subscriptions tables, the entity_id column will be added and amasty_example_cl will be written into the table (it’s created automatically for the work in the Update By Schedule mode):
Also, there is a version_id (AUTO_INCREMENT) field in the amasty_example_cl table. The field is compared to the version_id field from mview_state for a current indexer.
As a result, in our execute($ids) processor will get only those instances that have been changed after the previous ?rone launch (amasty_example_cl.version_id > mview_state.version_id).
How to set up Cron job in Magento 2 for working with indexers?
The main Cron jobs for working with indexers are defined in the vendor/magento/module-indexer/etc/crontab.xml file.
When launching Cron job by indexer_reindex_all_invalid (every minute), executeFull will be started for invalid indexers (they are marked in the Indexers grid with REINDEX REQUIRED status). It is intended for a full reindex, that is, in this case for all the invalid indexers, the table with the saved information will be rewritten.
When starting Cron job by indexer_update_all_views (every minute) all the indexers that work in the Update On Schedule mode will be launched. While launching it by indexer_clean_all_changelogs (every hour) the _cl tables (in our example it’s amasty_example_cl) indexers will be purged of entries with version_id below the latest version in the mview_state table.
Note: Check out how to reindex Magento 2 indexers manually without cron.
All the presented Cron jobs belong to the index group. The work of the Cron job group can be set up here Stores -> Advanced -> System:
Do you want to manage all the cron tasks in one place? Try Cron Scheduler! This module allows you to track cron tasks in a timeline and manage them in bulk right from your backend. Check it out!
How to manage Magento 2 custom indexers through the command line?
To see a complete list indexer commands in Magento 2, click the link.
Note: While executing the bin/magento indexer:reindex amasty_example command, you may face the error:
‘Amasty Example index is locked by another reindex process. Skipping.’
You can solve it in two ways:
#1. Either you have to wait for the end of the reindex process, which is now being executed;
#2. or the reindex process has been interrupted for any reason and the status meaning hasn’t been changed in the table, as the process wasn’t finished:
If this occurs, you can reset indexer in your Magento 2 via the bin/magento indexer:reset amasty_example command. It will change the working status to invalid and while launching the next reindex it will work.
That’s all for today.
If you have questions or suggestions, feel free to pose them in the comments below.
And stay tuned for new technical posts!