A/B Testing – How does it work and why do we need it?

Table of Contents

What is A/B testing?

In an A/B test, two versions of a landing page are tested: 'A', often called the "landing page" or "reference page", and 'B', an alternative version of page A.

During the testing process, one of these two versions is randomly shown to the target audience, and the traffic generated on the website is divided between the two versions, for example, 50/50 or 50/60.

The test for these two versions is run concurrently, which is important because it allows us to control as many factors as possible. It is also important that the random behavior of new visitors between the two versions of the landing page is equally distributed, as randomness is a key element in the calculation of probability needed for statistical analysis of the results.

I will show some advantages of the A/B test that I have observed

  • The A/B test has a simple structure, which is an advantage over multivariate or multifactor tests that require more detailed planning. All you need to do is decide how many versions of the page you want to test and direct visitors to them in the appropriate proportions.

  • The best version in the first test is declared the winner, so no validation tests are needed.

  • It is easy to implement with the help of available and free software like Google Website Optimizer. We can design and implement the test and collect data after a while, and use analytical tools like Google Analytics G4 to monitor traffic on the tested websites.

  • The analysis and conclusion of the A/B test are straightforward; we just need to compare the results obtained from version A with those from each new version and check if the intended results have been achieved. We can do this with simple statistical calculations.

A quick video guide about A/B testing