The web developer’s guide to A/B testing

Running a web development business isn’t an easy endeavor at all. You have to manage multiple projects, tough clients, and a constant stream of relentless deadlines.

One key methodology which enables a web designer to generate more return on investment for his clients’ websites, web ads, email newsletters, offers and promotions is A/B testing.

Don’t worry if the term sounds alien to you. It’s actually a very simple process; segmenting your users into two different groups and showing each group a different version of your page. To put it in a simpler way, A/B testing — or split testing as it’s sometimes known — is showing different page elements to different people and understanding which one gives better sign-ups or sales.

Why is A/B testing important for small businesses?

For one primary reason: to make more money.

As a web designer working with a small business owner, you know that investment by your client has a clear return on investment (ROI) expectation attached to it and successful conversion rate optimization provides one of the best returns on that investment.

To see how it works, let’s take a simple example. Suppose the website has 5,000 monthly visitors with a conversion rate of 4%, meaning 200 people end up buying. If you wanted to increase sales by 50%, you’d have to increase the number of visitors by 50%. Getting the additional 2500 visitors would mean expenditure on online/offline marketing activities that might or might not attract visitors who want to buy.

The second option is to optimize the conversion rate.

You think hard about what your client’s customers want, create a different headline that you feel better communicates the value provided by his business and split visitors 50/50 between the original (control) and the variation (also called “challenger” by some).

After a week when your test has had about 1000 visitors, the software says it’s confident that the variation headline is better and converts 10% more than the original headline. That’s 20 more sales at the cost of 30 minutes to sign up for an A/B testing tool and setting up a test. Not to forget the large smile you’ll be greeted with when your client sees you.

Image via Shutterstock

This is why the eCommerce world is raving about Conversion Rate Optimization (CRO). The return-on-investment for CRO, through A/B or any other form of testing can be huge. It has been in wide use by large enterprises like Amazon and Microsoft for years but has only recently become easy and affordable enough for web designers and small business owners to undertake on their own.

Choosing a tool

A variety of A/B split testing tools are available and for your initial test, it’s best to go with a free trial version. You can see a good comparison at WhichMVT.

What elements to test

The great part about A/B testing is that you can test almost anything. To get started, you should try some of the following:

• The Call-To-Action’s wording, placement, size and color
• Headline or product/service description
• Pricing and promotional offer display
• Images on landing pages
• Copy on product pages
• Insertion or removal of videos
• Amount of text (short versus long)
• Form length and types of fields
• Link overdose (too many and you run the risk of distracting visitors)
• Inclusion of Live Chat

This is obviously not a comprehensive list, but is very good place to start if you’re just beginning with A/B testing and want to concentrate on those elements that, in my experience, have shown to give the best results.

Setting up a robust test

It’s extremely important that you run the A/B test in a correct manner to prevent any error creep-in.

First, decide what it is you want to test and on what page of the website. For example, if the client reports that a lot of his customers ask for a certain feature which he does have, you could try mentioning it in the landing page headline. If a lot of visitors come to the signup page but then go to other parts of the website, you could try reducing link and text clutter and keeping the form as the focus of the signup page.

Test the control and variation on the same kind of visitors at the same time. For example, the visitor segment coming in from Facebook, in the week starting 1st July 2012 will be 50/50 split between a landing page with a control and variation headline. Avoid the common error of showing the control for a week and then the variation for the next. The comparison between their two conversion rates is erroneous.

Image via Shutterstock

Let the test run its course. Often, you’ll see a positive increase within a day or two and feel great about it. But hold your horses. There’s something called statistical confidence that tells you whether the results you see can be trusted or not. Wait for your chosen A/B testing software to report statistical confidence.

Don’t let the test run for too long. This may result in lost sales as you may be showing some of the visitors a low performing variation. Use an online calculator to decide how long to run a test, most A/B testing software comes with this kind of tool.

If you have changed something that is shown on many pages, ensure that visitors see the change across all pages. For example, if you have changed the color of the price image on one product, a particular visitor should see that same color across your product catalogue. A mismatch could mean inaccurate results.

Understanding the results

A CRO test can have three kinds of outcome; a positive outcome, a negative outcome or a “no result”, i.e. there is no difference between the control and variation. The first time you test, it’s likely that you won’t get a positive result. That’s OK. Testing to optimize conversion rates is not a one hit game. You must do it over and over again, tweaking every page on your client’s funnel to maximize the overall conversion rate.

Image via Shutterstock

Many times, it will happen that the results surprise you. What you might consider to be a low performing headline or website design might turn out to be the better performing one. In such cases, it is best to let go of your intuition.

A/B testing isn’t about following your gut; it’s about letting the actions of visitors tell you what’s best for your client’s business.

A few tips

When making changes, make it large. Don’t hesitate to make bigger, bolder changes. This ensures that you get varying results and better insight into what works and what doesn’t.

Headlines are super important. When creating them, work with your client to focus on the benefit that his customers derive from his products/services and not on what he wants to sell. Think from their perspective. Convince your client to test lots of different offers such as a free one month trial, three for the price of two, etc.

Put in a mechanism to collect customer and media testimonials and then add them in the website. These go a long way to providing social proof.

Keep the focus on benefits and not features. The client will probably love his product because of the awesome features it provides, but customers love products for the benefits they provide.

Test, test, and then test some more. After that, test again. Obviously, once your client starts to see a few positive results, you won’t need to do any convincing.

Conclusion

In today’s competitive business world, small businesses are using the internet to maximize value and quite often, they do this using the help of a freelancer or an agency. A/B testing is one way in which you can optimize your client’s marketing presence and generate more sales. For maximum effectiveness, this should be done in a continuous manner, ensuring that your clients see you as someone who helps their business’ bottom-line.

Have you introduced A/B testing to your clients? How did it affect your conversion rates over the long term? Let us know in the comments.

Siddharth Deswal

Siddharth Deswal works at Visual Website Optimizer, the world’s easiest A/B testing software. He’s been involved with web development for about 8 years and actively looks to help online businesses discover the value of Conversion Rate Optimization. He tweets about A/B testing, landing pages and effective marketing tips on @wingify