20th February 2020

A/B Testing: How to Make the Most out of Every Page

David Walsh

If your webpages receive a good amount of traffic but you know that more visitors could be converting, you can use A/B testing to improve the user experience and increase your conversion rate. In this post, we will explore the meaning of A/B testing and assess some of the most effective ‘quick-win’ A/B tests that you can try out yourself.

What Is A/B Testing?

You may have heard of A/B testing or split testing as a method of testing two different versions of the same page. A/B testing works by creating a controlled setting and a variation, and splits the website traffic that this page would receive so half of the users land on the controlled ‘A’ page, and the other half land on the variation ‘B’ page. From here, we can monitor site activity and conversion rate on each version of the page, ascertain whether the changes we tested were effective and, if so, implement these changes on future page designs.

How to Decide What to Test 

A/B testing can help you to test a hypothesis, but first you need a goal for the page you are testing. Then you can begin to formulate a theory about which elements would make an impact in order to form that hypothesis. Your theory may come from analytics data, such as on-page bounce rate, or from using an online user feedback tool like Hotjar. 

I-COM blog - AB test hotjar.png

Image: https://www.hotjar.com/tour/

For example, when analysing data from a Hotjar review, you may notice that users are not scrolling to the bottom of a web page. Perhaps they are not getting as far as the call to action and, as a result, the page has a poor conversion rate. An A/B test could help you to assess whether conversion rate increases if you move the call to action to a more prominent location on the page. 

Examples of A/B Tests 

You may have a theory that there are some design tweaks that need undertaking, but you don't want to budget for design updates across your whole website on a whim. A/B testing can help you to justify spending on certain page elements, as the data justifies the budget. 

You might want to run A/B tests on PPC landing pages, product pages, or even on your homepage. Here is a list of tests that you could run:

On product pages, you might test the following page elements: 

  • The leading product image 
  • The colour, size or position of the call to action buttons on page 
  • The syntax of your product title or description 
  • Filter options or the layout of filters on a page 

On a service page, you might test the following: 

  • The position of a 'call to action' message on a page
  • The position of the contact form
  • The prominence of the page title or key elements about a service

On a homepage, you might test 

  • Placement of different key messages
  • Proposed changes to a mega menu 
  • Placement or content of banners 


​​​​​Framework for Successful A/B Testing 

Below is a basic framework for A/B testing that you can apply to your website. 

  • Note down the goal that you want the page to achieve. For example, “I want to increase sales of product X” 
  • Make a list of variables that you want to check. This could be colours, buttons, text, the call to action that you include or its placement on the page 
  • When you’ve made the list of variables, think about your hypothesis. For example; “If I change the ‘buy now’ button from red to green, we could encourage more sales”
  • Test your hypothesis in practice by using A/B testing. You could employ third party software to help you out, like Google Optimize 
  • Run the test until the result is statistically significant. You need to have enough responses to demonstrate a noticeable difference between ‘A’ and ‘B’ to be confident of the direction you should take. You also need to be aware of conversion rate confidence, so you can be sure that your margins of error are different enough for you to say that the perceived impact of the change was real  
  • If the test returns meaningful results (i.e. an increase in conversions, or user experience improves), roll out the changes

Potential Pitfalls of A/B Testing

Split testing is the kind of activity that involves a financial investment as well as the time and expertise needed to formulate a hypothesis and run a test. There are two potential issues that you can run into with A/B testing, which you should avoid. 

Lack of Data Collection 

You need to ensure that you leave the test running for long enough to gain meaningful data. If you check the results too early, you may see an incorrect result and if you make any changes at this point, this will affect your data. Let the A/B test run and check the results at the end of the test before you make your decision on whether to roll out the variation. 

Individual Testing

One downside of A/B testing is that you have to test your theories individually to comprehend which elements are most successful. You may have grand ideas about four or five different elements that you need to test but you have to create these as individual tests. 
If you want to test more than one theory at a time, try multivariate testing. Multivariate testing allows you to test multiple elements, but it’s harder to analyse and track the exact changes that make a difference. 

How I-COM Can Help

If you’d like to conduct some A/B testing on your website and you are looking for expert support and advice, don’t hesitate to get in touch with our digital marketing experts today on 0161 402 3170 or by using the contact form