A Beginner’s Guide to A/B Testing

Looking for a low cost way to increase your business’ conversion rate? If you are, then it’s time to start A/B testing. Doing so can improve conversion rates significantly and, at the same time, refine your marketing message.

What is A/B testing?

A/B testing is a way by which the effectiveness of one variable is investigated. It’s also is a great way to let your inner geek shine as you put on your scientific lab coat and begin experimenting with emails, landing pages, paid ads, CTAs, and really anything related to your business’ digital marketing efforts.

An A/B test compares two versions of an item in which both versions are identical save for one variable.

An example of A/B testing
Let’s suppose your business wants to improve response rates to an email, and hypothesizes that changing the color of the CTA will draw more attention to it.

Two versions of the email are created with Version A displaying the Call to Action in the blue color you’ve been using, and Version B displaying it in a new red. Your audience is then divided evenly into two groups; half receives Version A and half Version B. After sending the email, you discover that 300 of those who received Version A responded to your CTA, but 500 of those who received Version B did. In this test, Version B was more effective that Version A, and is the new winner.

Focus your A/B testing

With beginning A/B testing it’s imperative to focus your test by creating a clear hypothesis and testing only one variable at a time. Not knowing precisely what you are testing for or making more than one change (creating more than one variable) will provide fuzzy feedback.

An A/B test is not the time to do more with less.

In upcoming articles, we’ll look at multivariate testing (as well as the importance of statistical significance), but for now, it’s one hypothesis – one variable.

What to test?

The only restriction on what you can A/B test is your imagination. That being said, there are common variables to test that will produce more for your proverbial buck. A/B testing is a great way to test:

  • Subject lines in emails: An effective subject line draws more of your audience into your email. It won’t matter how great the content of your email is if no one is inclined to open it.
  • Email headlines: Your email subject line encouraged your audience to open your email and a great headline can encourage them to visit your site. Find out which headline has the most power of persuasion through A/B testing.
  • Paid Search headlines: A/B testing is especially important in a medium where word count is extremely limited, and every word really needs to count.
  • Call-to-Actions (CTAs): A CTA is one of your most important tools. Find out which is most effective by testing CTA wording, color, and placement.
  • Promotions: A/B testing your promotions will tell you which offer is most appealing to your customers.
  • Layouts: Which email layouts do your customers prefer? One column or two? Which color?
  • Images: The types of images used all play a role in how effective your marketing can be. Test for placement, type of image, even subject, i.e. product or person?

A real life example of A/B testing

So what does effective A/B testing look like in the real world? Take a look at this A/B test performed for VONT client, VIP Tires & Service.

VONT wanted to find out if it would be more effective to put the VIP name or name brand Goodyear Tires in the headline of a paid search ad. Two versions were created with Version A carrying the headline “Goodyear Tire Sale,” and Version B headlining with “VIP – All Tires on Sale”.

The result? Version A resulted in a 1.18% CTR, while Version B resulted in a CTR of 2.36%. That’s a 100% increase in CTR over Version A. Needless to say, Version B continued to run.

The bottom line for A/B testing

A/B testing should be used on an on-going basis and is most effective when you use it to implement incremental changes.

Don’t look for the quick fix – one A/B test does not a digital marketing plan make. Think of A/B testing as a way to continually refine your efforts.

Once you have a winner, it becomes the new control for your next test to improve performance. A/B testing is a never-ending game of “Beat the Control.”


About VONT Performance Digital Marketing

At VONT we believe that change is the only constant in the digital world – and that excites us. When tools and environments are constantly changing, new opportunities to help our clients achieve success are constantly arising. Each new advertising technology, social platform, or design approach allows us to improve on the results we achieve for our clients.

We believe in this idea of continual fine-tuning so much that we named our company VONT, which means to achieve exponential improvement in incremental steps. It is our core belief, and the reason why we are not simply a web design company or simply a digital advertising agency, but rather a long-term, single source partner providing a comprehensive array of web development and digital marketing capabilities.

In short, we’re here so that our clients achieve success in the ever-changing digital world. If you’d like to learn more about VONT and the work we’ve done with our client partners, visit our Work page. Or, if you have a question, contact us. We’ll get right back to you!

Heidi McInerney
Written By

Heidi McInerney

UX Writer | Digital Content