Ho Ho Oh No — Do Searchers Care about the Holidays? | SEO Split-Testing Results

Brian Moseley

Dec 09, 20212 min read
Ho Ho Oh No — Do Searchers Care about the Holidays? | SEO Split-Testing Results

Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal.

First, we asked our Twitter followers to vote:

Peter makes a good point. We do always run our tests until we receive a statistically significant result, which is typically 3-4 weeks for highly trafficked websites. And there is always variance naturally, but typically we are focused on Google results only. Thanks, Peter!

The Case Study

Greetings once again, SEO aficionados! One of the areas we’ve covered a number of times before, and which continues to pay dividends over and over, is adding persuasive copy to your page title tags.

In this test, we’re working with an ecommerce company specializing in gifts and customizable products to determine if web searchers care about the holidays in such a stressful time. What do you think?

Merry Christmas Thumbs Up GIF by The Tonight Show Starring Jimmy Fallon - Find & Share on GIPHY

The Hypothesis

As we’ve noted before, persuasive copy is generally a pretty easy win, but it isn’t always, so we wanted to test first, of course. Our savvy team posited that adding “Holidays 2021” to the end of the page titles for a test cohort of category pages would result in a positive lift in clicks from search results pages. This would be because messaging was timely and captured searchers’ attention while shopping for gifts.


The Test

So, we set up the test using SplitSignal. A percentage of the category taxonomy pages within the site were chosen as the test variant, while an equal number from the same types of pages were chosen as the control group. We launched the test, and after only two weeks (normally, it takes double that), we reached a statistically significant conclusion!

The Results

SplitSignal ran this test over only 14 days, and quite happily, we observed a statistically significant, POSITIVE result for the client, with Google crawling and indexing 99% of the test group. In fact, there was a 17.2% increase in clicks to the test group! 


The end result is statistically relevant and is not only significantly reflected in increases to clickthrough rates but presumably to overall revenue attribution as well. Happy days!


Val is 100% right, way to go Val! Val’s insight was right on target, this company’s product is designed to be customizable and highly giftable, so this experiment turned out quite well for their clicks!

Did you guess the right answer? Quite simply, holiday messaging continues to be a significant motivating factor for consumers, which shouldn’t surprise marketers.

As we’ve often noted in these tests, we found not only an increase in clicks but also, as you would expect, a measurable increase to the test cohort click-thru rate (CTR).

So, where’s the magic here?

This particular test, much like several in the past, is very straightforward and depends on human psychology persuading users to click a message that satisfies their implicit need, regardless of its ultimate position on the search results page. 

We are easily influenced by persuasive copy, and in this instance, we’re putting a holiday-themed message in front of a captive audience searching for holiday gifts.

And the great thing about a tool like SplitSignal is that you don’t have to spend a lot of development or copywriting time to test your ideas. This holiday test took a few minutes to set up and just a little longer to deploy.

Have your next SEO split-test analyzed by the technical SEO experts at LOCOMOTIVE Agency.

Author Photo
Brian MoseleyGo-To-Market Lead for SplitSignal — Semrush’s new SEO split-testing tool which helps growth marketers understand which on-page SEO changes will drive the most organic traffic.
More on this