This article is designed to give marketers the correct perspective and provide practical instruction on how to run tests on search ads, display ads, remarketing, content marketing and customer retention marketing.
Ready? Let's go!
What Do Good Fantasy Football Managers and Marketers Have in Common?
The best fantasy football managers don’t always have the best draft; they don’t always start the player who has the best performance in a given week; they don’t always get the hottest player off the waiver wire; they don’t always get lucky...
OK — if they aren’t doing these things, what are they doing?
The best fantasy football managers let the data make the decisions for them. They optimize for performance by eliminating human error. They draft and start the players who have the highest percent likelihood to perform best over the course of a large enough sample (the NFL season). In fantasy sports, we are able to look at the projected performance statistics based on mounds of available data; the same is true with marketing.
If a fantasy manager makes decisions based purely on well-calculated projections, over the course of a 16-game season the likelihood of their team winning increases (as long as they have the players to out-project their opponents). In marketing, these same theories can be applied to maximizing ROI. Think of your acquisition channels as your roster and you need to know which “players” will perform best for a certain situation.
As the fantasy season wears on, the Yahoo! & ESPN weekly projected points become more reliable. Why is that? The sample size becomes bigger and the projected point totals have achieved statistical significance, thus reducing the likelihood of sampling error.
When Giovani Bernard went down in Week 9, you started Jeremy Hill and crushed it. When Bernard came back, you started Bernard over Hill, big mistake. Yahoo!’s projected point values were thrown off and countless fantasy managers also made that mistake. After seeing how well Hill performed, there was zero existing data to predict who would get more touches in the Cinci backfield. As the season progressed, our confidence level in Hill (the percent likelihood that Hill would outperform Bernard) rose dramatically. Essentially, we collected enough data to know that Hill would get more touches and thus be more likely to outscore Bernard.
How is this analogy relevant to marketing? You must establish which metrics matter and collect enough data on those metrics in order to make a well calculated decision… the more data you have, the more likely your projection is to be accurate. We call this eliminating sampling error.
Manage Your Digital Fantasy Team
Look at your acquisition channels like players. Each of them is taking up a roster spot (a chunk of your marketing time or budget). You need to make sure you are “starting” your players that are likely to perform best. In order to make these decisions, the associated “Likelihood” of performance needs to be statistically significant.
A good fantasy manager would never bench Peyton Manning after a bad game or trade Aaron Rodgers after a bad start to the season (R-E-L-A-X). Just like you shouldn’t cut your AdWords budget after a dip in performance over a short period of time. This is why statistical significance is so important. You need to be 90%, 95% or 99% confident that a marketing channel is or is not performing well.
Create Your Own Projections: Through Testing. Testing. Testing.
In fantasy football, we all have the luxury of A. a unified sample (we know what stats matter, we have previous seasons’ data to help us make predictions) and B. Yahoo! and ESPN have spent millions on their analytics platforms. Lucky us.
In marketing, we need to establish what metrics matter most, how to collect them, and determine how to run tests on statistical significance. When calculating statistical significance, it is open to the marketer’s choice, but I generally use one-tailed tests that calculate the p value. Five easy steps that can be applied to testing in general are:
- Establish what you’d like to know: “I would like to know which of these 16 ad variations has the best conversion rate for selling my green widget.”
- Make a Prediction
- Create variations
- Distribute to live environment
- Collect Data
- Run your calculations by using a statistical significance calculator
Which Areas of My Marketing Mix Will this Help?
Understanding which of your channels are performing and which are not is the key to figuring out where to spend your next marketing dollar, and there are varying levels of sophistication for doing this.
According to Cardinal Path Senior and Founding Partner, David Booth:
For some advertisers, a starting point might simply be tracking conversions for direct response SEM campaigns. For others, they're starting to integrate data sets and leverage the combination of traditional data science techniques with the enormous data storage and computing power now available for some very sophisticated cross channel and cross device attribution and media mix modeling.
- Search Ads (Part 1)
- Display Ads (Part 1)
- Remarketing (Part 1)
- Content Marketing (Part 2)
- Customer Retention (Part 2)
The decisions on what to test are all dependent on what makes sense for your business. However, there is a unified approach for thought; ask yourself, “What is the ideal ad to serve for purpose X?”
You can/should find the optimal ad copy to be served for the below purchasing scenarios:
The ease of collecting data within AdWords and Bing is dependent on the complication of the test you are running. Sometimes it’s possible to run these tests within AdWords, sometimes excel or testing tools are needed. Advertisers will need to set up the necessary campaign structure within the ad platform, TURN ON “Rotate Ads Evenly” and I would recommend spending at least $150 per day on your test. If you start to exceed 30 ad variations (or have lower volume keywords), more money will be required.
Cross campaign analysis is required for anything taking place at the campaign level. If you are trying to optimize ads for a given schedule group or geography, downloading campaign data and exporting for comparison will be necessary.
When creating this test, use your best existing ad as a baseline for the experiment. Pull out the metrics from your tests (Ad variation, variables within the variation, CTR, CVR, ROI) and compare the results to the baseline.
Again, using a statistical significance calculator can help you confidently determine which ad variations work best. Successful experiments will yield a statistically reliable choice for your ad spend. Tagging ads is very important. Using excel or another tool to filter performance based on the variables the ads contain is important to improved experimentation.
Impact on Your Roster
This is your quarterback. A lot of your acquisition is surely dependent on making search successful. Break off 5% of your search ad budget and dedicate it to testing. You will still be converting users during your test, so the budget isn’t wasted. You will also be investing in usable data and increased sales. Once you’ve found a winning ad variation, spend more on it. The go back and try to beat this winner with another test.
The goal is to find find a variation or “player” that maximizes that roster spot.
Display ads are not easily tested, but the extra work can pay off exponentially. Most marketers will tell you that display ads can be “assisters” and are not as valuable as search ads for providing direct conversions. I believe those marketers merely haven’t found the optimal ad combination for what they sell and who they are attempting to sell to.
You wouldn’t serve the same ad copy to someone searching for two completely different products and you definitely wouldn’t serve the same print ad to different target audiences. The theories behind display ads are the same, but marketers are either too lazy or don’t have the design capability to make iterations to their display ad creative.
A huge component of why display ads are so compelling to marketers is Google’s ability to predict who they are being served to. These targeting setting for display ads is what makes them so testable!
Variables Within a Display Ad (Content + Targeting)
Setting up data collection for Display ads is very similar to that of search ads, minus one thing, the ability to filter based on variable.
When you upload your display ad, you are uploading a flat image to AdWords or Bing, and not necessarily tracking which creative elements go into that ad. The process of importing your ads into AdWords requires a very detailed naming process. The names of your ads should tell you which creative elements are contained.
Impact on Your Roster
If you can find an ad that has improved your conversion or CTR to the desired level of statistical significance, then theoretically you have have an optimal player to “start.” Display ad testing can be a bit more difficult because of the design constraints, so think of this like your DEF or Kicker. You might have to rotate a lot of different ad variations until you find one that performs well.
More and more we are seeing remarketing achieve success. There are new ways to implement and test a remarketing strategy for almost all business types. The technology behind remarketing is a great way to obtain the required number of brand touches necessary to convert a purchaser.
Integrating Google Analytics with your Remarketing campaigns can provide an entirely new dynamic to your tests; the activity taking place on your site is now a variable in experiments.
According to Rocco Alberto Baldassare, Founder & CEO of Zebra Advertisement, here are some reliable tests that can yield positive results:
- Test different banner color combinations to see how they affect the CTR of your campaign.
- Test different remarketing audiences based on your Google Analytics data (e.g. test if people spending more time on site convert more once remarketed to). Here, our metric to evaluate is sales.
- Test different calls to action in the banners. View the impact on the CTR and eventually conversion rate.
- (in case b is too advanced) Testing variations of the landing page to aim for more conversions. Keep in mind remarketing users need a different landing page than the one they already visited and did not convert on!
What are the variables in a Remarketing Ad?
Fundamentals for Collecting Data
Isolating audiences (interest, demographic, location, etc.) by either campaign or ad group is imperative to holistic data. Similarly to both Search and Display, rotating ads evenly is the best way to collect enough data on every ad variation before making decisions. Also, exclude your ads from showing up in mobile apps, these are sometimes irrelevant clicks and will throw off the experiment data.
Again, using the calculations for statistical significance is imperative to making the right decision.
Impact on Your Roster
For certain businesses, remarketing ads can be compared to your Wide Receivers. Consistently providing touchdowns and big plays when you need it.
Part 1 vs. Part 2
In Part 1, we have reviewed how the skills you have cultivated as a Fantasy Football manager translate into your approach as a digital advertiser. These same theories can be applied to your Content Marketing and Customer Retention Marketing. Part 2 will be released next week by SEMrush.