What's Happening In The A/B Testing Market?
⏰ Old Post
This post is over 10 years old - it might be slightly outdated and consigned to the history books.
What is A/B Testing?
A/B testing is an approach to optimizing conversions on web pages which has become increasingly popular over the last couple of years. Most commonly used to increase conversion rates on sales pages or the purchase funnel of an e-commerce store, A/B testing gives a simple way to compare the performance of one page against another. Typically, two versions of a page are created which are identical in all but one aspect (perhaps the sales copy, or color scheme, or BUY button) and are shown to different sets of people. Each page has a goal, which is usually for a visitor to sign up or buy something, and the rate at which it "converts" visitors can be tracked. The performance of each version of the page can then be compared to work out which is most effective for encouraging visitors to take the next desired action.
This approach to testing has been around for a while now, particularly amongst some of the biggest online stores (for example Amazon, Walmart and Netflix). However, until 2010/2011 the process for carrying out the tests was quite complex, which made it inaccessible for many site or store owners. 2010 saw the launch of several tools specifically aimed at simplifying A/B testing, and presenting results clearly and effectively. Since then, A/B testing has really taken off online with countless sites now using the method in an attempt to squeeze out every last bit of efficiency. The use of A/B testing has grown at an impressive rate over the last three years, with the main reason for its growth being the introduction of easy-to-use, affordable tools to help do the job.
With that being said, let's take a look at some of the popular A/B testing tools out there.
Tools of the Trade
As the chart below shows, there are eight popular A/B Testing technologies being used to optimize websites today.
A/B Testing Distribution Top 1 Million Sites July 2013
Whilst there is no single technology completely dominating the market, Optimizely and Visual Website Optimizer account for almost 50% of websites carrying out A/B testing. They are clearly the big players here, so let's take a look at each in a little more detail.
Visual Website Optimizer
Visual Website Optimizer first appeared on the scene in beta form during 2009 with a full version being launched in the middle of 2010. The uptake of Visual Website Optimizer was incredibly quick and it rapidly overtook numerous other A/B testing tools which were available at the time. Within a six month period from October 2010, Visual Website Optimizer went from relative anonymity to being one of the key players in the market, second in usage only to Omniture Adobe Test and Target. After the initial upward trend, things plateaued for a year or so and the tool struggled to increase its market share. However, since last summer the upward trend has restarted and if anything, looks like it could be picking up pace.
Visual Website Optimizer is more popular amongst smaller sites, claiming more than 18% of the market for sites in the top million but only just over 10% for sites in the top 10k. None of the sites using Visual Website Optimizer have a Quantcast rank of less than 5000 and the tool seems to appeal mainly to smaller ecommerce stores and websites.
Optimizely
Optimizely was co-founded by Dan Siroker and Pete Koomen who had both previously worked as product managers at Google. It was Dan's experience in another role though that inspired the development of Optimizely. Dan served as the Director of Analytics during the 2008 Obama presidential campaign, where he used A/B testing to raise an additional $60m in extra donations. He was frustrated by the complexity in running simple A/B tests and with the tools that were available, so decided to do something about it.
Optimizely was launched around the same time as Visual Website Optimizer in 2010 and has seen steady and consistent growth ever since. It's now the most popular A/B testing tool on the market.
Unlike Visual Website Optimizer, as the chart below shows, Optimizely has a consistent market share amongst big and smaller sites.
Visual Website Optimizer vs. Optimizely Market Share across 10k, 100k and Top Million
Some of the big names using Optimizely include skype.com, vimeo.com and time.com, so it's clear they're doing something right. It's the major players like these that some of the other A/B testing tools, including Visual Website Optimizer, are struggling to add to their fold. This only goes to reinforce the message which is proudly displayed on the about page for Optimizely: "Optimizely is the #1 website optimization platform in the world."
What's Next for the A/B Testing Market?
The A/B Testing market has grown from virtually nothing over the last few years. The growth has been quick and shows no sign of slowing down. As the chart below shows, the usage trends of the two tools discussed above is quite clearly in an upward direction.
Visual Website Optimizer vs. Optimizely Market Share in Top Million Sites since January 2011
A closer look at the rest of the data suggests that these two tools are not growing at the expense of other tools on the market, but instead that the market itself is growing. There are a lot more sites using A/B testing technology now than ever before. Whilst the market is growing so quickly, there may well be enough room for all of the currently available tools to build their user base. This sort of growth can't last forever though. As things start to slow and competition for users increases, it wouldn't be too surprising if we begin to see the tool providers merging with each other or the bigger players buying out their smaller competition.
For a closer look at the data behind these A/B testing tools, as well as a whole host of other technologies, head over to the BuiltWith Trends pages. If you'd like to dig a little deeper, accessing a richer dataset and compiling customised reports, BuiltWith Pro might be for you.
Discuss this post on Hacker News.
