Why Is A/B Testing Imperative for Ecommerce?

Gut Instincts and A/B Testing Build Ecommerce Websites


What Is A/B Testing?

If you have never heard of A/B testing, don’t worry, as it’s merely jargon that refers to evaluating which of two options works better. The first option is the “A,” and the second one is the “B.” It is actually that simple. This article will tell you about my experience with A/B testing, it’s advantages, and some words of caution.


“Should We Present the Homepage By Category or Brand?”

Recently an ecommerce entrepreneur who retails industrial supplies asked me this question. While horizontal B2C ecommerce players usually display home pages with categories, this entrepreneur suspected that his buyers were actually more interested in knowing what supplies specific brands had to offer. My response was, “Your guess is as good as, or probably better than, mine. But why guess? Why not allow the customer to tell us.”


And this is the classic use case for A/B testing:

  • you are evaluating two different approaches to a specific issue
  • the issue is important, and you cannot afford to screw it up
  • you have an opinion about which approach would work, but are not confident enough to adopt it
  • it’s possible to inexpensively set up your website for each of the approaches

If the above description strikes a chord, then you might be ready to try out your own A/B test.


What Can You Test?

There’s a lot that can be easily tested using A/B testing. Here are some examples:

  • Should you use a long product page, or are short snippets more appealing?
  • Should I have a drop down menu even though the drop down is too large or should I allow a drilldown left menu?
  • Does a small “Buy Now” button on the top right work, or should I put a huge button right next to the product image?
  • Testimonials occupy valuable screen real estate. Should I include above the fold, or push them lower down on the page?
  • High-resolution custom images that can be magnified further will require spending some extra money. Are they worth it?

If you have spent any time in ecommerce, you know that these are important questions that need to be answered. But you also know that these are a small subset of the large number of similar questions you need answered when it comes to content, navigation, experience, and architecture. Think about it, how much would you be willing to pay to get high-confidence answers to such questions?


How Do You Go About It?

When you set up an A/B test, make sure that you:

  • run both experiments simultaneously, otherwise it might turn out to be a case of comparing apples and oranges.
  • only vary one parameter instead of having several things different between you’re A and B. Otherwise, you wouldn’t be able to establish causality.
  • use cookies to ensure that the same visitor doesn’t see different versions of your website on subsequent visits.
  • make a judicious decision on how long to run a test before arriving at a conclusion.

Keep in Mind, This Is Only a Test

Garbage-in-garbage-out was never as true as it is in A/B testing. Despite your best attempt, your test might be flawed. So make sure that you are using your gut feeling. Many would baulk at what I just said, but I think that no test can substitute experience and expertise a seasoned ecommerce professional brings.