Resolve split opinions with Split Testing

Julia thinks the ‘BUY NOW’ button should be green; Tim says orange, like Amazon. Does this kind of debate sound familiar?

Split testing (often called A/B testing) can get the real answers to questions like these.

Here’s how it works:

• Make both versions — one page with orange and one page with green.
• Randomly show site visitors one version or the other.
• Track these visitors and measure which works best.
• Go forward with the better version.

It really is that simple.

The key point of split testing is that you are trying all the versions at the same time. This is the only way to get results you can trust. Trying A in January and B in February is not the way to go: there are lots of other differences in visitor behaviour and traffic between January and February.

There are different ways to do split testing. You need to think about:

• Keeping the site consistent for each visitor: when she comes back to the site or back to the page she should see the same version.
• Getting statistically valid results: If A works for 2 customers, and B works for 3 then it’s not 50% better. You need to wait for more traffic.

You are not limited to just 2 versions: you could test several versions at once in an A/B/C/D/… test. This is rarely done; it’s much more common to test a changed page against the existing page and see if the changed page is an improvement. The change can be a series of pages: for example, A is a 3-page checkout process, and B is a 2-page process.

A more advanced idea is multivariate testing. You change several elements at the same time and run the tests. These changes could be different colours, different graphics, and different copy. They could also be changes to the offer: different discounts, free shipping, free upgrades.

As you can imagine, multivariate testing is highly complex and only found in more enterprise-class testing software. A major consideration of multivariate testing is not so much the software but the human effort needed to derive all of the potential design permutations you want to test. For example, testing 3 elements, each with 3 alternates to test would be mean the creation of 9 elements to test in all. For most web page optimisations, A/B testing is more than sufficient and a huge step forward.

The great thing about split testing is that it is practically unlimited in its application. Just about any single aspect of your website can be tested to help you optimise your web pages and continuously improve your website’s conversion rates.