Hello to Amasty blog readers!
With this article, we’re starting a series of blog posts devoted to Magento testing. Sometimes it feels like this topic is not covered in full when it comes to the Magento community. We’d like to fill in the holes here!
Our first article is written for those who are developing and/or testing their own Magento extensions. We’ve compiled a checklist of the basic things you need to test before releasing any tool, free or paid. These points are quite basic and are aimed at people which are just starting their journey in the world of Magento product testing.
What is more, this checklist can be of great help for Magento merchants. Using some of these points, you can test if the newly bought extension is working fine on your Magento store.
1. Make the list of influences the extension possesses
It will be helpful to write a test plan for your extension.
- Does it work on backend, frontend, or both?
- Does it work with prices, products, customers, orders and other groups of features? List them all.
- Does it work with third party entities, such as payment options, third party extensions, or any other services? List them all.
2. Test the extension functionality
Use specifications or other inner documents to check all the module’s options.
Pro tip: If you're a merchant, use Magento extension product pages, user guides and demos for reference.
3. Test Magento features which can be influenced by the module
Distribute them into several categories: payments, taxes, catalog management, customer management, etc. Use a Magento features list for reference.
4. Test your Magento extension on several platform versions
You probably know that Magento users don’t stick to only one Magento version.
This is a chart from 2016. Right now the tendency has changed only when it comes to the popularity of certain versions, not the pattern in general:
Via BuiltWithIdeally, you have to test the module on each Magento version that is available. In reality, you choose which Magento versions you’re going to support and, consequently, to test your extensions on. It is recommended to test Magento 1 extensions on versions from M1.7 to the latest relevant one, for Magento 2 from 2.1.o to the latest relevant one.
Pro tip: with time, you’ll see that there’s no need to test the same functionality on all the versions (say, 1.7.0, 1.7.1, and so on) because there's not much difference between them (which could matter for the particular feature testing process), and QA guys usually are aware of the changes that were made from version to version. At Amasty, with several years of testing we worked out a system based on which we’re testing the extensions.
We’re picking three versions. The first one is the latest stable one, the second and the third ones are the older versions which are popular and have the most difference between them. If we do know that the extension can work with areas that were changed in the particular third-level versions (say, 1.7.0, 1.7.1, and so on), we also add them to the test plan.
This allows us to save time and to cover the most differences when it comes to various Magento versions.
So, think before testing! Build a plan and compare versions to incorporate them into your schedule.
4. Test your Magento extension on various designs
If the extension makes some changes to the frontend, you should test it out! Check the Magento out of the box themes (the old one and the new one which comes with 1.9 and later versions – RWD (Madison Island) first! Don’t forget to cross-check the mobile and tablet view in case of RWD.
5. Test if your extension supports website/store/store view entities
Make sure that all the settings which incorporate store view choosing don’t influence the other store views. Also, global settings (which should be applied for a website) are influencing all the store views. This is an important Magento peculiarity which should be considered while building a test plan.
6. Load test your Magento extensions
Test your extensions, if they will potentially deal with big amounts of data or visitors. Check response time of the pages with high traffic on the frontend and how the extension handles big chunks of data on the backend. For example, any import and export options must be checked on how they handle big files.
This is a very important point, while, of course, everything on this list is important as well. While backend speed hiccups or issues influence store management effectiveness, frontend high load issues can lead to sales and revenue loss just like that.
Pro tip: to make load testing convenient, we created some sample Magento installations with a lot of stores, categories, orders, customers, products (as well as product images), product attributes etc, and we employ tools that can load test pages to check their response time under high visitors traffic.
7. Cross-browser checking. Web, mobile, and tablet if supported
This is a basic test for any website – check how the pages affected by the extension are performing in Chrome, Safari, Firefox and Internet Explorer.
If the layout supports mobile and/or tablet, or your store has a separate mobile version, make sure to see how the pages perform on various screen sizes, Android and iOS and in the couple of popular mobile browsers.
Pro tip: don’t forget to cross-browser test the backend, too.
8. Check how installation and deinstallation processes go
Also, don’t forget to include a deactivate test into your plan.
9. Check how the update process goes
If you’ve made some substantial changes to the module structure, it’s necessary to know if the client is able to update any of the previous versions without any trouble. Pay special attention to the possible data and settings loss.
10. Check extensions compatibility with third party extensions and services
First of all, you need to think about the list of extensions you’re going to make your product compatible with. Usually, if you’re working in a team, your Project Manager gives you this information.
We at Amasty are testing all of our extensions for mutual compatibility (130+ Magento 1 modules and 70+ Magento 2 modules). It’s a hard part of the testing, but it is not as complicated as it could be because when we’re checking a module, we know that all the previous products were tested. In other words, it’s easier to identify and fix a bug when you know for sure which product is causing it.
Pro tip: have a copy of Magento with all the extensions that should be mutually compatible installed. Thus, performing other tests on such a Magento installation will reveal all the compatibility issues right away,
11. Perform the standard security check
This applies for Magento extensions which work with files and data (import and export, forms, and such) and includes file opening from a direct link, permissions, form validations, and such. This is a big and important topic, which will be covered in detail in our future articles.
12. Test in Production mode
While it’s not an issue with Magento 1, testing in production mode on Magento 2 has some peculiarities. If you install the first extension onto a clean Magento 2 site, the production mode turns on automatically. If you need to install a second extension, you have to turn the production mode off, install the extension and turn it on again.
13. Cross-check the above points
For example, if you’re checking two Magento themes with your extension, apply cross-browser testing to both of them.
So that’s the basic list of the points you absolutely have to incorporate into your Magento extensions testing routine.
We will cover more of these points in our future articles. Please let us know if a particular part is more interesting for you, and we’ll schedule the next articles according to demand.
Thanks, and stay tuned!