This article is a preface to Jason’s upcoming SEMrush webinar Introduction to Advanced Multivariate Ad Testing. This article introduces some of the concepts that will be discussed: reasons why multivariate tests need to happen, how to execute these tests, and good fundamentals for testing.
This webinar will take place on December 15th at 2pm Eastern time. Jason is the CEO of AdBasis, an advanced multivariate testing platform for Search, Display, Remarketing & Mobile ads. Jason tested over 2 million ads in 2015 and has taken the time to speak with us in order to educate our audience on what his webinar will provide.
1. What is multivariate ad testing and when should it be used?
Every search ad is made up of variables... not only the content (headline, display URL, descriptions, landing page) but also the targeting (who sees the ad, what they searched for). Multivariate ad testing is an analysis of the elements within an advertisement, and it should be used when multiple elements within a single ad variation are up for debate.
Advertisers need to select which combinations of elements work well together for specific audiences, products and keywords; multivariate testing is usually the best way to draw these conclusions.
2. Why would someone need to run a multivariate test?
Great question. If you’ve identified areas of your AdWords account that are underperforming, a great way to get performance back to your averages is through testing. There may be certain elements within those ads that just aren’t working well, and using multivariate tests to identify those elements is a great way to solve these problems.
The ultimate goal of a multivariate ad test is to Find the Optimal Combination of Ad Content + Landing Page for Each Situation That Makes Sense for Your Business. If you don’t have an optimal ad for each “purchasing scenario,” that’s a sign that you probably need to start testing.
3. What are some good fundamentals to know if you’re getting started with ad testing or if you’re pushing a new initiative that needs to be tested?
Good fundamentals for ad testing revolve around two main categories: planning and execution. There are definitely some key things to remember in each of these categories that will help you be successful.
Use Your Marketing Calendar: Your marketing calendar should dictate your testing schedule. Your upcoming promotions, product launches, corporate initiatives will tell you exactly what needs to be tested.
Plan Ahead: More often than not, advertisers have tons of work to do. Every day seems to get away from us with overwhelming workloads. By the time a promotion hits, the opportunity for testing and optimization is already over. Creating a testing calendar 3-6 months in advance will really improve the likelihood that the optimizations will be executed to their fullest potential.
Types of Tests: Use a Z-test for CTR and Conversion Rate. Use a T-test for CPA and ROI.
Minimum Sample Size: Gather at least 100 clicks per variation before making decisions on CTR. Some advertisers may look at this and say “That’s way too small” which may be correct. But it depends on the brand, if you a smaller brand with relatively low volume, be sure to gather at least 100 clicks per variation before making a decision.
Distribute Tests Across Multiple Ad Groups or Campaigns: In my webinar, we will show you how to do this. Increasing the number of ad groups in your test allows you to aggregate data and speed up your experiment results. Through aggregation, you may be able to achieve statistical significance in an area with relatively low volume.
Shut Down Your Sub-Optimal Ads ASAP: As soon as you’re able to uncover an ad which is outperforming your baseline to a degree of statistical significance, spend as much money on it as soon as possible. Doing so will maximize the improvement to ROI that you’ve acheived through your test.
Join the Webinar
Please join me for my SEMrush webinar, Introduction to Advanced Multivariate Ad Testing. This webinar will take place on December 15th at 2 PM ET. Click here to register.
Jason Puckett is the Founder and CEO of AdBasis, the world’s first testing and optimization platform for Search, Display and Mobile Ads. Directly prior to launching AdBasis, Jason served as the Vice President of Growth for SocialKaty, Inc. which was acquired by Manifest Digital in 2014. Jason’s specialties include digital ad optimization, sales growth and all things digital.