If you haven’t read Part I of this series, this entry is only going to make half sense, so I would highly recommend that you read it. In our first installment, we discussed the importance of making data driven decisions in both your Fantasy Football management and your marketing mix. We reviewed how the theories behind statistical significance will impact your advertising decisions for search, display and remarketing.
Part II is designed to help marketers develop the correct testing mentality and provide a usable process for data collection. What questions do we have to ask ourselves in order to determine usable solutions to content and retention improvements?
- What are my key performance metrics for both content and retention marketing?
- What tests can I implement to collect the right data?
- How do I determine if this data is reliable enough to make decisions?
Fantasy Football Scoring and Your KPIs
When you play Fantasy Football, your commissioner is nice enough to determine these metrics for you; standard scoring, PPR, maybe some Special Teams points ... easy stuff. To make it even easier, Yahoo! and ESPN have spent millions of dollars building analytics systems to help you forecast your projected point totals and keep track of the metrics that matter.
Digital marketers aren’t so lucky, but the same mentality can be applied. Determine your marketing equivalent of a Rushing TD and use projections to determine which “players” should fill your “roster spots” (in Part 1 players= marketing channels, roster spots= marketing resources). Again, if you’d like to read my comparison of Aaron Rodgers to your AdWords account, go back and read Part 1.
Yahoo! Creates Projections Based on Data. So Should You.
OK, so after we’ve asked ourselves the above questions, we now need a plan for accumulating data for our projections. Here is a general process that can provide success to any testing plan:
- Establish what you’d like to know- “I would like to know which of these 16 content pieces has the best conversion rate for lead generation.”
- Make a prediction
- Create variations
- Distribute to live environment
- Collect data
- Run your calculations by using a statistical significance calculator
Of the elements discussed so far, content marketing is definitely the toughest to measure. The key is establishing which metrics make a difference for the conversion type you are driving toward. Of course, revenue is the number one concern, but sometimes content marketing is so high in the funnel that it is difficult to attribute any piece of content to a specific sale. It is important to generate a custom metric for your content marketing and measure only a single metric per test.
According to Uri Bar-Joseph, the Senior Director of Marketing for Simply Measured:
“To me it gets down to the four components of content marketing.
- Distribution Channel
The problem with measuring the success of content marketing lies in the fact that it has these four components that can have equal impact on the overall success...since the four components are so tightly tied, in order to measure them separately some A/B testing is necessary. The key to accurate A/B testing is to hold all variables constant while testing only one of them.”
As we know, the use of a statistical significance calculator will help marketers determine if the data in their experiment is usable. Comparing a variation to a baseline is easily conducted once the metrics have been established and the test has been conducted.
Here are some good examples of A/B tests that can be conducted for Uri’s above four components of content marketing:
A/B test the titles of your PDFs to see which generates more downloads. Exposing the content to a big enough sample to reach statistical significance can be tough. Make sure you are sending to the same number of email addresses or using the same dollar amount in Facebook ads budget too.
Test the schedule during which your content is being distributed. Content marketing is all about exposing material to an audience organically. Maximizing reach is crucial. Take notes on day of week and time of day that your content receives the highest engagement (clicks, downloads, likes, shares, etc). We have found that Tuesdays at 12 p.m. are our ideal time to post B2B content.
Take the same piece of content and reformat it, track the success. Sometimes the same piece of content will work better in a different media sometimes transcribing audio interviews into written blog posts will gain more traction. A video lecture may have more visual appeal on Slideshare. These tests can provide tons of value but be careful, changing the format may sometimes change the promotion channel and will throw off your testing data.
Impact on Your Roster
Content marketing is the Julian Edelman of your marketing mix (if you are in a PPR league). Not very much flash, but can provide reliable points every week and assist in your wins.
Customer Retention Marketing
Customer retention is not exactly in your acquisition funnel but needs to be talked about. CRM is a form of marketing that is designed to cross-sell, upsell or resell your existing customers. Even if you are a SaaS company with monthly recurring charges, you need to resell. Don’t kid yourself; every month your customers are making decisions whether or not to repurchase your product.
Finding specific tests and the associated results that can improve retention by even a few percent can have the same impact as increasing conversions with significantly less overhead.
There are Four Main Things to Think About Before You Run a Retention Test
- What you send- It could be a survey, a thank you, a coupon, it could be a small new product. Something to touch your customers and build loyalty.
- How it’s delivered- Direct email, email, in-app, etc.
- Audience- The subset of customers will receive this impression from your company. Are they people who have been customers for six months? Are they brand new customers coming up on their first bill?
- Metrics to Measure- How will you know if your retention campaigns are working? Example metrics include: percentage of customer subsegment that their subscription that month, percentage of customers who come back to your site, etc.
Send A Positivity Booster Right Before Billing- Whether you are sending a coupon, an exclusive piece of content or a free trial of a members-only feature, make you it’s a real value add to the customer.
What you’ll need:
- 500 or more current customers (or as many as possible)
- Two days/times to send the promotion
- One format of the content
Only send the customer retention program to half of the audience. The other half will receive nothing. Distribute your retention marketing efforts during your two selected days/times (half of customers receive promotion one day before and the other half receive it hours before billing). You will then see if sending a this promo improved retention, and the optimal time to send it.
Introduction Message- Send half of your customers an introductory message or give them a call. Note which do and do not receive this touch. Measure the overall retention rate or LTV for clients in each segment. We have seen studies where this can go either way.
Stepping up Your Game
Whether your number one goal is beating your college roommate in Week 16 or driving revenue dollars to hit your quarterly marketing goals, I hope this article has provided a deeper understanding of statistical testing and how it can improve your process and management. Always remember to keep the data in mind when making decisions and always work to improve the sample of data you are pulling from.