18 Top A/B Testing Tools Reviewed by CRO Experts

By Mahmoud Omar Last updated Apr 17, 2021

A/B testing is no longer a new field. Finding a proper A/B testing tool isn’t the problem anymore. Now, the problem is choosing the right one.

If you work in conversion optimization—whether at an agency, in-house, or as a consultant—you almost certainly run A/B tests.

You could hire someone full time to sift through and analyze the pros and cons of each tool, but it’s easier to learn from the experience of others and make a decision based on that.

To help you with that, we’ve assembled a list of the most popular A/B testing tools and corresponding reviews from A/B testing experts to help you in your decision making process.

Without further ado, here are the top A/B testing tools used by experts (click below to navigate to a specific tool):

AB Tasty

AB Tasty is a reasonably priced, simple-to-use tool that serves as a good starting point for companies just starting in conversion optimization.

AB Tasty offers A/B testing, split testing, multivariate testing, and funnel testing capabilities. Using AB Tasty’s visual editor, you are able to quickly put together variants and tests and receive real-time reports with confidence levels on your goals.

At the same time, you have full control of testing through advanced targeted scenarios, allowing you to trigger tests based on URL, geolocation, demographics, and so on. You are also able to enrich the platform’s data by integrating existing data from CRMs and DMPs.

Chloe O’ Keeffe, former associate conversion consultant at Conversion.com, shares some pros and cons:

Chloe O’ Keeffe:

“Pros:

The interface is really clear to understand and launching or monitoring tests is easy. There are a lot of available configuration options, for example you can add multiple filters such as user IP address, localization or cookie values.
Test set up is easy. The editor is easy to use with a useful preview mode, and users with little technical experience can easily modify pages and create new designs or use the unique library of widgets.
Many integrations are available and useful for validating test results, such as Google analytics, Clicktale or Kissmetrics.
Pricing is mid range—not as expensive as other leading competitors in the market.
Customer service team are very responsive and quick to tackle problems if any arise.

Cons:

The statistical significance calculator is somewhat basic; rather than a numerical value this is displayed as a bar chart.
There can occasionally be a lag in test results which requires a response from the customer service team and can delay the testing process.”

Pricing for AB Tasty is available on request.

Adobe Target

Adobe Target is a popular enterprise tool that combines targeted testing and personalization.

Adobe Target walks you through a three-step workflow where you first create the variant, then target the variant based on visitor segmentation, and lastly customize your goals and settings for the test. Based on this emphasis of targeting, it should come as no surprise that Adobe Target’s most exciting feature is automated personalization.

Target’s self-learning algorithm continually and automatically adjusts itself to make associations between visitor behavior and content in order to give you the best results possible. Real-time adaptation of content can help you better meet the needs of diverse visitors.

Here’s a review from Sean Patterson, former senior conversion consultant at Conversion.com:

Sean Patterson:

“An enterprise testing tool with a wide range of capability and customization. It has similar features to other enterprise testing tools such as Optimizely, including personalization and recommendation modules for more advanced optimization.

The real power with Target, as with all Adobe Marketing products, is when they are integrated through the Adobe Marketing Cloud. Segments uncovered in analytics or through their data platform can be pushed to Target, and results from Target analyzed within Analytics for increased insight.

As with many Adobe products, integrations can be complex and require subscriptions to the other tools in the Marketing Cloud to be most effective. As with Analytics, additional modules for Target such as personalization and recommendations come at a cost.”

Pricing for Adobe Target is available on request.

Apptimize

Apptimize is a testing tool focused on mobile optimization. It’s a perfect choice if mobile is an important platform for your business.

Apptimize offers you a full-control visual editor where you’ll be able to rapidly create new variants and target them at different user segments. You’ll also be able to add dynamic variations and code blocks that support Objective-C, Swift, Java, HTML5, Xamarin, and React Native. Their randomized experiment methodology also ensures your experiments will return statistically significant results.

We sourced reviews for Apptimize from G2. Here’s an anonymous review talking about some things they liked about Apptimize:

Easy to get set up and use, awesome to be able to create tests without updating the browser (after the initial Apptimize-enabled version is deployed).

And here’s a review discussing some disappointments:

The real-time changes that we can deploy are actually limited in the scope—so, not all of our original goals can be met with this tool. Also, we have to plan in advance what sections of parts of our app we would like to change, and only after that we can start using this feature, which means that there is a lag/delay between the time that we want to change something and the time when it actually becomes a reality. For example, with our sprint structure this delay can be as long as three to four weeks.

Apptimize starts at $995/month and supports up to 100,000 monthly active users. You can also request a custom plan based on your needs

Conductrics

Conductrics is an advanced tool that offers methods from A/B testing blended with machine learning to delivering optimal experiences, for each user, automatically.

You can use it as either a server-side or client-side tool, and you can pretty much customize it in any way you want to use it.

Here’s a review from an experimentation and optimization expert Andrew Anderson:

Andrew Anderson:

“Pros: Offers a lot of flexibility as far as ways to interact with your systems, from APIs to one tag solutions. It’s at its best when it is not being used for just straight A/B testing, but instead as an active decision agent that is constantly adapting and creating experiences.

Because of the ability to select factors instead of just blindly adding them all in, it allows for this functionality on far smaller sites than other similar tools (Adobe Target Premium/1:1/Touch Clarity, X+1). Very robust system with very low variance and a wide variety of features. Great support, although the help menus are a bit technical for most groups.

Cons: Requires more knowledge about decisioning agents than most programs have. Interface is a little clunky with the current page interaction set-up rather annoying (though this is supposed to be fixed soon). QA is less than ideal. Data reporting can use some improvements, especially graphically. Has a bit of a learning curve when it comes to reading the Targeting report. Still requires a fair amount of traffic to reach full potential.

Overall: Conductrics is a great tool for programs that really want to tackle adaptive learning personalization or want a lot of flexibility when it comes to interactions.

It does require a lot more knowledge and more of a learning curve then very basic tools like Optimizely or VWO, but in the end can provide far better functionality and more flexibility than just about any other tool on the market. Great tool for those programs that are above the standard tools but not at a Fortune 100 size.”

Pricing for Conductrics is available on request.

Convert

Convert is perfect for small to medium-size clients and businesses doing in-house optimization. It is well loved for its exceptional customer support over live chat.

Convert operates using an intuitive drag and drop system. You can create A/B, multivariate, multipage, and split tests using the provided visual editor and break out more the more advanced style sheet editor for dynamic pages, giving you full control of your content.

It also boosts a smooth onboarding process of your CMS and ecommerce platform and seamless integration with Google Analytics. Personalization is also an option with over 35 different elements you can use to build customer profiles.

Here’s Julien Le Nestour, an applied behavioral scientist and CRO consultant based in Sydney, giving his opinion:

Julien Le Nestour:

“Convert is a fantastic tool that should be considered and evaluated by any CRO agency or end-user client that also evaluates Optimizely and VWO. They offer a level of functionality that is globally on par with them for testing, but at a lower price level once you go above the free level offered by Optimizely.

The Convert team is very experienced in testing and they go to some incredible lengths to offer an unparalleled amount of support when you are building or running your tests. In my experience, they will assist you in all technical aspects and issues and solve them for you as part of their plans.

You can also ask them to directly create tests for you if you don’t have anyone on your team to do it as an additional service. The only (potential) limitation in using Convert would be a more traditional approach in measuring statistical significance, but whether or not it’s an important point for you depends on your views on this subject!”

Convert’s most basic plan, Essential, starts at $599/month and gives you access to 10 active goals, 5 personalizations, unlimited tests, advanced targeting and segments, and more. You can also contact sales to explore enterprise plans.

Dynamic Yield

Dynamic Yield is a powerful suite of personalization and engagement solutions.

With Dynamic Yield, you’ll have access to a wide range of tools. Most importantly, you’ll be able to continuously A/B test and optimize across platforms (desktop, mobile web, apps). These tests are predictive in nature and leverage the capabilities of machine learning algorithms to continuously improve ROI.

Dynamic Yield, however, specializes in advanced omnichannel personalization solutions. You’ll be able to segment and quantify every user interaction and response and dynamically adjust your content to best suit each individual. Combine your segments with personalized notifications to get the most out of this particular tool.

According to a review by Martin Nicholson on TrustRadius, some pros of the tool are:

The out of the box features are impressive and quick to implement. Overlays, messaging and content changes can be made instantly enabling tests to be carried out quickly.
Creation of custom actions enables us to bridge the gap between our online and offline tests by injecting code changes directly onto the page.
The test segmenting features are very robust, actions can be triggered by device, IP, cookie, audience type, and specific times of day, allowing us to test whatever we want, when we want.

And some cons:

We’re not using all the features of the tool, there’s so much to choose from it’s sometimes overwhelming. The support team however are superb and have always been able to help.
More integration with GA would be useful.
Some of the features do require some technical knowledge to implement, again support are always on hand to help.

Pricing for Dynamic Yield is available on request.

Evolv

Evolv is a new generation testing tool that merges machine learning algorithms with conversion optimization.

Evolv enables you to do goal-oriented experimentation. Once you’ve set up your goals and KPIs, it uses machine learning to experiment with the largest number of ideas possible.

Evolv covers the entire customer journey and can optimize elements like copy, button styling, background images, slider treatment and timing, image order, and more. One of biggest advantages of Evolv is the ability to run multivariate tests on a massive scale.

Pricing for Evolv is available on request.

Google Optimize

Google Optimize is a Google tool that lets you test and tailor different variations of your website.

The main selling point of Google Optimize is its native integration with Google Analytics, allowing you to gain automatic access to rich behavioral insights. You’ll also be able to target the valuable segments you’ve already discovered using Analytics.

Here’s what Charles Farina, former director of growth and development at Analytics Pros, has to say:

Charles Farina:

“Google Optimize is a completely free A/B and multivariate testing enterprise testing tool. With a visual editor, code editor, and native integration with Google Analytics it is the best tool for companies that are just starting with testing. Many nice-to-have features are still missing, like an image upload feature, but Google continues to roll out enhancements every week. You have to use or install Google Analytics to use Google Optimize.

I’m excited for the next few years, as Google Optimize should disrupt the testing tools market the same way Google Analytics did when it was released and started to get more and more enterprise features added.”

Try out Google Optimize for free and upgrade to Google Optimize 360 for enterprise-level website testing and personalization.

Intellimize

Intellimize is a predictive personalization platform that automates many of the more laborious and time-consuming aspects of setting up and running A/B tests.

Using machine learning, the Intellimize platform tests personalized site changes, then automatically prioritizes the winning result.

Gabe Meierotto from LaserAway reviews it below:

Pros:

Typical A/B testing doesn’t come with development. With Intellimize, they develop ideas for us (HTML/CSS/etc.), freeing up our developers for other tasks and/or freeing up my time for high-level ideation and research.
We learn more quickly than a typical A/B test. Machine learning can tease out which variations are performing best more quickly.
Personalization: Intellimize will show different images/messaging to different people. Their machine learning automatically discovers the right set of experiences to show each individual visitor, feeding the winners more traffic and starving the losers of traffic automatically. For example, their system will show a hero image of ladies walking on the beach to people in California and ladies in the city for people in New York.
They help us test better ideas by sharing their experience from seeing thousands of experiments.

Cons:

Some ideas take longer to develop than we would like.
We have to generate reports for our team manually.

You can try Intellimize by contacting the company via their website.

Kameleoon

Kameleoon is an advanced optimization platform that offers a variety of features.

Using Kameleoon, you’ll be able to run advanced A/B, split, and multivariate tests quickly and easily. You’ll also be able to understand key user insights with their navigation analysis tool and test different user segments with over 40 targeting criteria.

Dynamic traffic allocation algorithms also make sure your traffic is optimally divided in order to shorten your decision cycle and improve return on investment.

Julien Nouet, omnichannel platform business lead at Kering, reviews it here:

Julien Nouet:

“We first started using Kameleoon to A/B test our website to make sure our clients were happy with the experience we offer. We chose Kameleoon because of its reliability for testing at high velocity and its advanced features.

The platform met our growing needs as we also now run personalization campaigns, not only to improve our website performances but to create tailored experiences for our visitors.

Features we appreciated:

Top notch anti-flicker technology
Strong and innovative architecture (NoSQL, Elasticsearch)
Works with React
Integrates well with our other solutions
Powerful targeting features
Ease of use of their WYSIWYG editor
Click tracking easy to setup
Test results can be filtered directly (by device for example)
Possibility to have a both visit or visitor-based test samples

Some cons:

WYSIWYG Editor a bit slow to load
Some elements missing in the preview recap (capping, script setup…)
UX has some room for improvement
Kameleoon teams are innovating a lot… And the platform may be down for shorts periods of time. No biggie.

We’ve been working with the solution for more than a year now and we’ve had a great ROI. Overall, it’s a strong optimization platform both quite simple to use and feature-rich. As a side note—not immediately related to the tool but, their customer success team is worth mentioning. They are quick to respond and provide valuable strategic insights.”

Get started on Kameleoon by creating a free account and then upgrade to an enterprise plan if the tool suits your needs.

Maxymiser

Maxymiser is a powerful testing tool that helps you design and run sophisticated campaigns.

Like other tools, Maxymiser offers a visual editor that will help you set up variants and tests as well as an extensible coding environment for more complex projects.

However, Maxymiser shines through most in terms of personalization and targeting. This tool provides you with the ability to build rules directly using the visual editor to target profitable customer segments and display personalized content.

Most impressively, Maxymiser automatically builds a customer profile that aggregates data on historic and in-session behaviors and visitor attributes for each unique visitor to your page.

Pricing for Maxymiser is available on request.

Monetate

Monetate is another well-loved testing tool that has a reputation for providing fantastic client services. They provide both a testing and segmentation tool as well as a personalization tool.

With Monetate Testing and Segmentation, you can perform advanced A/B tests and multivariate tests. They also have built-in measures of statistical significance and dynamic testing which adjusts traffic allocation to optimize for ROI. Monetate’s 1-to-1 Personalization Engine is an advanced machine learning engine that determines which actions will help you achieve a given goal. This means you can automate and optimize personalization decisions.

Pricing for Monetate is available on request.

Omniconvert

Omniconvert is another important player in the testing tool market. They position themselves as a sort of all in one optimization tool: on-page surveys, A/B testing, personalization, overlays, segmentation.

They’re an especially good tool for beginners, affordable with lots of features included.

Here’s a review from Justin Rondeau, Digital Marketer‘s general manager:

Justin Rondeau:

“What I like best is that this product does more than just A/B test. Qualitative data is crucial, and the tool helps you gather this vital information. I also like the personalization aspect of the technology.

Overall, Omniconvert captures where the market is going, but still uses A/B testing as a backbone to validate results gathered by its other components. I think the user interface could use some tweaking, maybe work on a more logical flow for finding what you need.

I am always trying to squeeze more productivity out of a page, whether it means more sales or leads. I have been able to make strategic design changes with the testing tool and gather user data with the survey functionality.”

Omniconvert’s plan starts at $324/month. There’s also an enterprise plan available.

Optimizely

Optimizely is an experimentation platform aimed at enterprise customers exclusively. Their tool is powerful and targeted at websites with high traffic—in other words, companies that are very serious about testing and personalization.

At the root of it, Optimizely is focused on web experimentation and personalization. With their Web Experimentation product, you can safely run multiple experiments on the same page.

You can also use Optimizely beyond your website and expand to mobile apps, messaging platforms, and more so you can optimize your entire customer experience.

Pricing for Optimizely is available on request.

Qubit

Qubit is a testing platform focused around personalization, making it one of the strongest segmentation tools on the market.

With Qubit, you’ll be able to run A/B tests and multivariate tests in order to measure progress and effectiveness of various personalization techniques. You’ll also get access to cart abandonment recovery, product recommendations, and social proof tools, making this software a great choice for ecommerce businesses.

According to a review on Trustradius by Niall Totten, some pros of using Qubit are:

Great A/B test ideas based on both quantitative and qualitative evidence.
The Qubit consultants and engineers we have worked with have an excellent ‘can do’ attitude.
Excellent communication and clarity on test results.

From the same review, the cons are:

There is room for improvement on the velocity of getting new tests developed and QA’d.
I would like to see improvements to the exit survey interface to make it easier to flag, categorise and disseminate feedback to different areas in the business.
Depending on the test/experience it can add extra weight to page load times and cause ‘flicker.’

Pricing for Qubit is available on request.

SiteGainer

SiteGainer is a company that offers a full suite of conversion optimization tools, including A/B testing, multivariate testing, personalization, heatmaps, popups, and surveys.

The tool is quite easy to set up and use, and creating everything from simple A/B tests to surveys, popups, and more is very intuitive.

They also have a team of experts to assist you with test ideas, design, programming, analytics, and personalization.

SiteGainer offers a free trial. Pricing plans are based on monthly traffic, payment schedule, and additional services.

SiteSpect

SiteSpect offers a server-side solution and is a great choice for more technically advanced testers. By editing HTML before it leaves the server, it is popular with self-hosting companies and security-conscious clients.

Of course, SiteSpect does offer a visual editor tool to help make setting up optimization and testing processes convenient and efficient. What makes it different is its ability to give you access to both client and server side, allowing for a comprehensive view of the entire customer experience and high performance for speed and scaling.

Here’s a review from Judah Phillips, an analytics expert, consultant, and author who wrote Building a Digital Analytics Organization, Digital Analytics Primer, and Ecommerce Analytics:

Judah Phillips:

“SiteSpect is one of the global leaders in testing and optimization. They are pioneers in the space with a decade of experience. Their patented technology provides the ability to test and measure testing results without having the change code on your website or mobile experience. The technology pattern matches patterns the customer defines and matches test content based on pattern recognition in real-time with no latency and total failover.

Features exist for speeding up the performance of tests and running many different tests simultaneously. Unlike many testing vendors, SiteSpect supports all markup languages (HTML, WML, JSON, and XML), style sheets (CSS), or scripting languages (like JavaScript). Any type of static, dynamic, and rich content, including audio and video can be tested. They were the first company to offer a comprehensive mobile testing solution.

While SiteSpect used to require specialized hardware, the capability can now be deployed in the cloud or on-premises. Like other vendors they have figured out the “flicker” problem and continue to innovate mobile testing. Detractors of SiteSpect report that the product has technical complexity and isn’t something that can instantly used by a marketer without engineering support.

It’s also a tool that requires people to understand what they are doing and plan for testing. The reporting interface has been criticized but is widely regarded as industry standard. Finally, it’s not a tool for people who want to spend $49 or more per month. It’s a powerful software for enterprises that are serious about testing and optimization and want technology that can accommodate for enterprise requirements under an SLA.”

Choose from a variety of packages that vary in integrations, features, and accounts depending on your scale and needs.

VWO

Visual Website Optimizer (VWO) is another big player in the A/B testing software market. It is a top choice for businesses with slightly smaller budgets and popular amongst many marketers.

One of the reasons it’s so popular is because of the variety in tools it gives you access to, which VWO separates into five categories: Testing, Insights, FullStack, Engage, and Plan.

With VWO, you are able to create multiple variations of web pages and track various conversion goals. VWO also provides statistical analysis of your results. A favorite feature of VWO is its built-in heatmap tool that tracks visitor click behavior and tracking habits.

Notably, if personalization is important to your work, VWO offers easy tools for segmenting and delivering targeted offers.

Here’s a review of Alex Harris, CRO director at Perficient:

Alex Harris:

“VWO is an excellent conversion testing platform. One of the main pros is the visual interface. In my opinion, I think the UX is more user friendly and the reports are more visually appealing. The reports and charts are easier to analyze than other testing platforms. Another good thing about VWO, is that is very easy for anyone to walk through creating a new test. It is a simple step by step process that I think makes more sense, especially to someone who is new to A/B testing. Previously, I would would have recommended another A/B testing tools if you were a developer, but VWO recently improved their code editor which in the past was really buggy and caused more headaches than needed.

The only downside that I have ran into is related to the WYSIWYG, which a good optimizer shouldn’t really use anyway.

In the past, when I didn’t use the code editor, the WYSIWYG was really buggy. For example, I would create a test and if I didn’t make a copy of the code that was created, sometimes the variation would break and I would lose all my work. This may not be the case anymore since I only use the code editor now, but it was rather frustrating when I lost my work in the past.

Also in the past their support has a slow response time related to tickets created for support.”

VWO plans for web-based testing range from $99/month to $467/month. Their FullStack plan is $1,999/month. A 30-day free trial is available.

Conclusion

There you have it: the top A/B testing tools, reviewed by the experts.

While each has their own pros and cons, it’s likely you’ll have to some some strategic thinking and exploration to get the solution that is right for your business and specific situation. Hopefully these reviews help you in the process.

Source: https://cxl.com/blog/ab-testing-tools/