In the melting pot of A/B testing case studies that you can find on the web, not so many of them are helpful for your business niche. One successful case can’t suit all industries due to the factors like diverse audience, sales funnels, and other things.
Actually, there is an approach that really fits every industry - prove your reputation with hard evidence and build customers’ trust. Magento 2 Product Attachments will help you.
We prepared a list of the most striking A/B tests that were run by or for e-commerce entities. We’ve grouped them by 6 topics for your comfort so that you can skim through the article easily.
So, let’s go on the tests.
Navigation experiments
Test #1: Navigation Bar
Business and its aim
FSAstock.com is the only e-commerce store that stock Americans with FSA*-eligible products and services. The company faced the problem: their visitors were hardly converted into customers. The hypothesis: there were too many distracting elements and options on the category page, so people could be confused with them.
What was done
An overwhelmed sub-header was removed from the site navigation.
The result
53.8% increase in revenue per visit on the category page.
Takeaways
Navigation bar was invented to simplify our life but not to complicate it. Lots of tabs could distract attention and confuse your potential customers. Back away from the monitor and look at your website: is everything clear enough at first view? Are there any elements that make you dizzy and are they actually necessary? Anyway, if any doubt runs through your head, it means the time to test has come.
*FSAs are employer-based programs that allow consumers to set aside tax-free dollars to purchase medical products and services.
Source: Big Book Experimentation by Optimizely
Test #2 Widget with footwear sizes
Business and its aim
Modlet is a Romanian footwear retailer which was dissatisfied with their 1,5% conversion rate. A hint was taken from the Google Analytics: it was noticed that those visitors who made a purchase browsed less than those who didn’t convert.
The key factor was the filtering option: people who used a shoe size filter made a purchase faster as they didn’t spend time on checking if their size was available.
What was done
It was decided to add a layered navigation widget for footwear sizes. By clicking on a right size, a user is redirected to a page with the footwear of the selected size.
The results
The shoe size widget has increased conversions by 20.5% and boosted the revenue by astonishing 68.9%.
Takeaways
1) Before brainstorming A/B testing ideas, look through your Google Analytics first.
2) Mind all decision-making factors when building your sale funnel. In real life, we select the pair first and then ask for a size. But filtering by size on a page is much more convenient, as shoppers won't face a situation when they've chosen a pair and found out there's no needed size of it in stock.
3) Direct the attention to the important things. A vast majority of sorting and navigation options usually simplifies the shopping process, but at the same time, it distracts. By highlighting the most significant sorting criteria, you help your visitors to concentrate on the right options and as a result - to navigate much faster.
Mobile magic
Test #1 Buttons colors
Business and its aim:
Extra Space Storage is a US-based self-storage provider with over 1,000 storage facilities across 35 states.
What was done
The original CTA button and several elements like star ratings, telephone icon, and ‘show map’ text were blue. The hypothesis was that if the CTA button color was different from the other text on the page it would spur users to convert.
Three CTA button colors were chosen for the test:
- Original blue button (control)
- Yellow button (not shown here)
- Orange button (winner)
The test ran only on the mobile version.
The results
The decision to change CTA button color was right. But if the yellow button performed a shade better, the orange had won by far: reservations were increased by 7.8%.
Takeaways
In western culture, orange is a color of fun, haste, and impulse. That’s why it can be especially effective on mobile where users make a fast decision on the go. Also, orange is often used for inexpensive products so it can be subconsciously associated with a low price and a benefit. In our case, the same color for a CTA-button and a “First-month free” text may be associated with a double bargain.
So how to choose the right button color?
Well, you can start to spy on your competitors for inspiration. Also, one of the hints you can get is the data of the best performing colors for CTA buttons. But even these hacks can't guarantee you the 100% result. In sober fact, dozens of factors matter: your website design, the audience and the business niche are just some of them. Moreover, don’t forget to keep in mind that various cultures have different color perceptions and associations.
Test #2 Splash page on mobile version
Business and its aim
Revolve is an online clothes retailer, which aim was to increase the number of mobile app users.
What was done
The team hypothesized that a splash page on mobile version with aggressive promotion could raise more downloads.
The results
The effect was astonishing: aggressive promotion increased downloads by 350%.
Takeaways
Despite all prejudices about the pushy advert, it still works. But to succeed you need to send a clear message. In the second tested variant, a customer can easily perceive a nice deal: “Take 15% off” text jumps out at your eyes right off the bat. In the first variant, the purpose of the message and the potential benefit of the app installation is not so evident. That's why a shopper prefers to skip the page than to try to grasp the meaning of the text. Anyway, you can try splash pages on your store if you think it will work for your audience. But mind that to succeed you need 2 things: a clear deal and a CTA button.
Page structure art
Test #1 Homepage
Business and its aim
Susty Party creates bright and colorful disposable eco-friendly tableware. Their website attracts enough traffic, but the visitors are hardly converted into customers. For this reason, the company set a goal to increase engagement and get more sales out of this traffic.
What was done
A high bounce rate on the homepage showed a lack of a clear USP. To improve the situation, several changes were made:
- real customer testimonials were added;
- product videos were removed from the home page (it was hypothesized that they distracted visitors’ and impeded them to get to the product page);
- a security affirmation was added to the checkout page;
- a CTA button was added to the banner.
The results
The conversion rate was increased by 250% from the homepage
Takeaways
The first variant lost as it didn’t provide a visitor with a hint of what to do next on the homepage. Instead of moving through the sales funnel, customers’ attention was tied down to the videos. Sometimes in pursuit of high engagement and stylish design, you can miss sales. Think twice before you decide to reduce ‘working’ elements like testimonials, catchy headlines, texts, navigation or CTA buttons in favor of chic layout. Like in our example, the lack of clear USP and CTA button costs too much.
Test #2 Mobile Landing Page
Business and its aim
Elegant Steps is a wedding shoe store. The company was dissatisfied with a low conversion rate from mobile devices - only 0,6%.
What was done
- To help customers find their way around, the ‘Shop by Brand’ section was placed higher on the page;
- The 'Free Shipping' text was displayed above the fold to encourage visitors to go ahead. According to researches, unexpected shipping costs can significantly increase cart abandonment rates;
- The CTA copy was changed from the 'Shop Wedding Shoes' to the 'Find my new wedding shoes';
- The text color on the image was changed for a more readable one.
The results
Conversions were increased by 3 times when the bounce rate was 50% decreased.
Takeaways
In this case, the elements like a bad combination of colors, a generic CTA title, and a hidden 'free shipping' note all together have provided a weak user experience. Again, it’s all about the way you present your unique selling proposition. If it’s vague or hidden, potential customers will be only your guest visitors, who surf the web stores just for ‘passive shopping’: look, dream, but don’t buy. Build the right relations with your visitors at the beginning by showing them not only fascinating product pictures but also the purchasing conditions and deals.
Price display hints
Test #1 Boxes versus Rows
Business and its aim
RS Components is a distributor of electronic, electrical, and industrial components. Their team suggested that the price area of the store wasn't easy enough to understand it right away.
What was done
It was decided to simplify the price table by redesigning the boxes to the rows.
The results
The number of clicks on the “Add to Basket” button was increased by 9.16%.
Takeaways
It's a challenge to organize complicated pricing options. And the only way to know how to do it right is to test! It all depends on how the majority of your audience perceives information. In this case, people who buy technical components are used to easier price rows instead of fancy tiles.
Test#2 Zeros in prices
Business and its aim
The test was launched for a jewelry fashion brand. The hypothesis was that the price length associates with the product cost: the shorter the price is, the cheaper the products seems.
What was done
All zeros after the decimal point were removed for all products on the website.
The results
The first variant has gained:
- 9.3% increase in add-to-cart clicks
- 29% rise in visits per order
- 47% lift in revenue per visitor
Takeaways
Well, the hypothesis was proved. According to the studies, price display has a great impact on customers’ purchase decisions. Zeros after a decimal point can make a product look more expensive. If you would like to emphasize a good deal, you can add zeros to a strikeout old price and reduce them from a new price.
Product page tricks
Test#1 Additional options
Business and its aim
Zalora is an Asian fashion retailer, that decided to optimize its conversion funnels. The team had found out that their return policy display on the product page should be modified to make it more noticeable.
What was done
Two changes were made:
- The 'Free' keyword was shifted from the right to the left position;
- ‘& Free Returns’ copy was added next to the price tag.
Four variations were tested:
- Control
- Variation 1 – Shift 'Free' to the left
- Variation 2 – Add ‘& Free Returns’
- Variation 3 – Shift 'Free' to the left and add ‘& Free Returns’
The results
Variation 1 had won and increased the checkouts by 12,3%
Takeaways
1) It’s all about a natural reading order in most languages - from left to right. Put the most important information on the left if you want your customers to read it first. Also, don't forget to mind a language of your audience: if your customers speak Arabic or other languages, where the syntax is vice versa, - the words order is inverse.
2) In this test, three 'Yes' writings in green boxes performed better than two of 'Free' and one of 'Yes'. It shows that a similar text in one of the variations was more noticeable and, as a result, worked well.
Test #2 The model
Business and its aim
AdonisClothing is a US-based e-commerce store, that is specialized in men’s fashion. But! The main store’s audience is mostly women who shop for their husbands and boyfriends. With relatively high product page views, 'Add to cart' rates were low, so that the company decided to optimize their product page.
What was done
Once the store owner was asked to grow a stubble by his fiancee that’s how the idea to test selling options of the 'beard' was born. Here's how the variations looked like:
The results
'Beard' variation defeated the Control by 49.73%, resulting in a 33% increase in orders.
Takeaways
As we see, a beard fashion continues to be a trend. Praise it or damn it, we can only suggest if the main trendsetters are women who literally pay their compliments mostly to the bristled guys (according to the test results) or the mass media and the global vogue. Anyway, girls unconsciously choose not an item itself but the image of their ideal man in it. So what we can learn from this case? Explore the tastes of our audience and keep a close watch on tendencies and fashion changes. Maybe some of them could bring you fresh ideas.
Test #3 CTA Transparency
Business and its aim
Evan Cycles is the largest UK retailer of bicycles, which online store has 1.5 million unique visitors per month. Once the company decided to improve their UX, it conducted a research by asking their customers for feedback. It was found that when some customers tried to buy a cycle, the item they chose seemed to be out-of-stock even if there were plenty of them available.
What was done
In the tested variant, the team made the “Add to basket” button brighter and suggested selecting size and color.
The results
The conversions were boosted by 49% and the revenue was increased by 4.3%
Takeaways
The logic is quite simple: remember how many times you saw that a product picture or any other attribute turn pale if an item is out-of-stock? Clearly, this pattern has been formed by web by now. Also, sometimes we just can’t identify the problem due to the lack of a detached view. Don’t be afraid to ask your target audience about their opinion and run a focus-group to get valuable feedback.
Headlines glory
Test#1 Copies
Business and its aim
California Closets’ offers premium custom storage solutions. The company was on their way to optimizing their sales funnel.
What was done
The team decided to change the heading and the subheading on their landing page to match the PPC advert copy identically.
The results
The relevant copy beats the “catchy” headline. The version with ‘You’ve Arrived’ headline increased submissions by 115%.
Takeaways
Сopy relevancy really matters. Artful and smart headings are an essential part of a sales funnel, but don’t rely on one text only. Create and test-test-test multiple landings before you find the best variant. But remember: your message should be clear.
Test #2 Navigation menu copy
Business and its aim
The test was run for Anton’s Cleaners - a dry cleaning company, that would like to increase their sign-ups for loyalty programs.
What was done
The copy of the loyalty program tab on the top navigation menu was changed from 'VIP Express' to 'Save 50%.
The results
The second copy increased a click-through rate by 37%.
Takeaways
The 'Save 50%' text offers an immediate value when the 'VIP Express' phrase can confuse the first-time visitor as its meaning is not clear. And one more thing. ‘VIP’ sounds expensive, doesn’t it? By changing the text to the more money-friendly one, Anton’s Cleaners found out what their customers really wanted: they just looked for a discount. Therefore, the winner was not just a new catchy copy, but its meaning. Make sure that all your tab and menu names express the idea!
We hope that some of the examples above have inspired you to work hard and experiment with your website diversely. Researches, data, surveys, customers’ feedback, competitors analyses or team’s ideas are important, but it’s not the whole A/B testing story. Grab ideas from everywhere and select the most feasible variants, that you can substantiate by reliable data.
And what test results were the most exciting for you? Share your thoughts in the comments below.