A/B Testing Your Mobile App? 4 Awesome App A/B Testing Tools Not To Miss

Jason Hawkins

Oct 20, 20165 min read
A/B Testing Your Mobile App

In the marketing world today, creating business apps is growing in popularity and A/B testing for webpages continues to be important, but A/B testing your apps is just starting to come to the surface. Now keep in mind, if your app is very new with a small number of active users per day, you likely have larger priorities than to configure an A/B test for your app.

However, if you’re in the triple digits of active users per day, there may be some small, simple changes that can be tested to improve user engagement and the value your app provides.

The sooner you can start running these A/B tests the faster you will see results that will help push towards your goals. Fortunately, there are A/B testing tools specifically for business apps that can help.

What Should I Test and Why A/B Testing Tools Matter

Unfortunately, businesses are A/B testing the wrong things all too often and it can take a while to realize this misstep. There are two great ways to get started finding test ideas that work:

  1. The first option is to schedule interviews with several of your users to collect feedback that could inform a design or UX improvement. It’s important to stay in tune with their needs and goals for using the app and interviews are a great way to do so.
  2. The second way is to review your app analytics or heatmaps. Where are users dropping off? Are they using the app as intended? What user segments or behaviors are more likely to convert?

Asking these questions can help you determine what you need to test to help you reach your milestones. At this point you’re ready to use an A/B app testing tool to test your hypothesis and really dig into your data to create the best user experience.

Once again, there are two reasons tools are important to use for A/B testing.

First — making changes to apps usually requires technical expertise, but now many simple changes can be made through some visual editor features of many A/B testing products.

Secondly — with these tools you can deploy changes to apps instantly without submitting a revised version to the app store and waiting for the user to make the upgrade. You can learn more about business apps in general by contacting our marketing company here.

Four Hot A/B App Testing Tools to Help

Below are four of the most popular A/B app testing tools on the market today:

1. Apptimize


A relatively new mobile app testing product out of the San Francisco Bay area, Apptimize is destined for growth after recently raising $18 million in venture funding.

Their A/B testing really emphasizes targeting the app onboarding process. Getting users to download your app is a challenge in itself but getting them inside, signed up and understanding how the app will benefit them quickly is an entirely different challenge. The step from downloading to becoming an active user is often where you see the largest drop-off, and their platform can quickly get you started with testing.

User flows are an important ingredient to a good app. What screens and functions should be available from each screen? Too many options and the app is confusing, too few options and the app doesn’t meet the user’s needs. This is a difficult task for UX and an optimization challenge but Apptimize has a solution. Here are some of their examples of great mobile app checkout flows.

Their visual editor makes it easy to swap out text, images and calls-to-action. Most importantly, quick and iterative testing is made possible by infrastructure that offers app releases automatically.

An additional benefit is that implementation is quite easy. In fact, iOS apps require no additional code. Their documentation is very clean, concise and thorough. As a marketing agency. this is the resource we typically use.

Integrations: Localytics, Mixpanel, Omniture, Google Analytics, or Flurry.
Supports: Objective-C, Swift, Java, HTML5, Xamarin, React Native.
Pricing: Free up to 10k monthly active users or a custom solution

2. Optimizely


Optimizely is the most “famous” A/B testing tool out there since setting up shop in 2009, but they didn’t have a tool for mobile apps until more recently. If you’ve ever used Optimizely, it offers a very similar user experience. 

The visual editor is great for making simple design changes like text, images and layout while the developer tools allow you to implement entirely new app features.

What really makes their app A/B testing service pop is their analytics. They offer lots of simple data visualizations for metrics like retention rates, unique conversions and your chance of beating the original (with 95% confidence interval).

A downside is that all of Optimizely’s products, web experimentation, web personalization, Optimizely X, and mobile are packaged together. There doesn't seem to be a la carte pricing available if you only need one or two of the products.

Integrations: 30 technology partners
Supports: iOS SDK and Android SDK
Pricing: Pay as you go $49/1k users or $999 month, Custom

3. Mixpanel


Mixpanel is a robust analytics platform for web apps and mobile apps that is trusted by thousands of companies. One of the features of their product is A/B testing. Their targeting is pretty neat, especially if your app is a game.

Using information about your users already in Mixpanel, you can run your experiment on specific groups of users. For example, you can make your game harder for all users after level 6 or change the UI for people who use a certain feature more than others.

In that same light, Mixpanel has a very clean and user-friendly user experience for setting up your tests and measuring them. Similar to Optimizely, if you’re just interested in A/B testing your app but not interested in a full suite of products like mobile surveys or setting up email notification to your users, it may be hard to justify the costs if you don’t qualify for the free plan.

Integrations: NA
Supports: iOS, Android and Full Stack
Pricing: Free up to 25k data points per month, then tiered pricing

4. Splitforce (recently acquired by Localytics)


Splitforce is an alternative that specializes in mobile app A/B testing without any other products. Their features make A/B testing very easy and user-friendly. Updates can be made instantly for both iOS and Android devices. Targeting is simplified to a handful of the most important options such as device type and behaviors like the number of prior visits.

One standout feature is auto-optimization. This algorithm automatically adjusts and assigns more traffic to a variation that appears to be winning instead of waiting for the test to run to set all traffic to the winner. This helps arrive at a winning variation faster.

While the product excels in its simplicity of testing elements like text, numbers, colors, buttons or custom designed elements, there’s limited opportunity to code advanced functionalities, onboarding or user flows for app A/B testing.

Integrations: NA
Supports: iOS, Android, Unity and Javascript (coming soon)
Pricing: NA

What A/B testing apps have you used in the past? Have you used any of the apps mentioned above? Share your thoughts and stories in the comment section below.

Mobile App Insights

Get access to Mobile App Overview and SearchAds Intelligence

ADS illustration
Author Photo
Jason HawkinsJason Hawkins is the CEO & Co-Founder of The Miami SEO Company. He has over ten years of experience in search engine optimization, conversion rate optimization and lead generation. His core responsibilities include identifying ways to increase value of services rendered, training staff on advanced SEO topics, and A/B testing internal processes to consistently improve client return on investment.
Subscribe to learn more about SEO
By clicking “Subscribe” you agree to Semrush Privacy Policy and consent to Semrush using your contact data for newsletter purposes