Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
Only 33% of our followers guessed it right! The result was negative.
Read the full case study to find out why.
The Case Study
Business intelligence, sales prospecting and competitive insights are all keys to growing a business. Since Semrush has a shared mission of helping businesses grow, SplitSignal partnered with one of the largest B2B providers of company and employee contact information. In this experiment, we tested title tags with 2 key changes.
The Hypothesis
Our hypothesis states that since many users are looking for employee information, adding terminology that connects to that user intent would lead to a higher CTR and thus more traffic. As part of this experiment, we also removed the brand name from the title tag to allow for more room to fit in the word “Employees.”
The Test
We used SpitSignal to identify 4,919 pages that would be part of our test. We utilized 2,471 of those pages as the control group and then executed a change on 2,448 variation pages. To make sure changes are seen in SERPs, SplitSignal also measures whether Googlebot has seen the variant pages since launch. In this case, Googlebot visited 97% of the variation URLs.
Our test optimization consisted of adding the word “Employees,” as our hypothesis stated that this was part of the reason people are searching for these pages. We also removed the brand name in order to afford room for the addition of the word “Employees.”
The Result
The test ran for 21 days from June 22nd through July 13th. This title tag change had a negative outcome, resulting in 3,695 (-6.2%) fewer clicks for the variant group compared to the control group. This test result has a 96% confidence level, indicating a strong degree of confidence in these results.
Why This Test Was a Winner
Sometimes, what’s not in a title tag is more important than what IS in the title. In this case, a brand that people know is hugely valuable.
For anyone searching company info, the name of the website is a familiar and trusted one. Removing this brand name reduces users’ ability to quickly parse out, “Hey, this is that brand I know and trust” from the SERP and means they do more work visually to note the source of the page (i.e., looking at the URL/domain). Less familiarity and a wordier title tag, we think, resulting in fewer clicks.
When looking at the last 12 days of the test we see that the CTR for the control group was 1% higher.
The phrase we added was also not one that these pages primarily rank for. At first glance, you may think that the word “employees” is exactly the type of term people are looking for when searching for these pages. However, “employees” only appears in 0.04% or 2,700 of 6.5M keywords these pages rank for, according to Semrush’s Organic Research tool.
The primary driver of traffic (and rankings) for these pages is the exact company name that the page is about. Adding language that searchers may not be after also may have led to a lower CTR. Similarly, having more phrases in the title may have potentially diluted ranking signals for what is ultimately the “primary” keyword the page ranks for.
The two key insights here are that if your brand has any type of strong recognition, and this one does with nearly 100K MSV, it’s important to include that name in your title tags. Conversely, if people are not actively looking for a term that appears in your title, you may not want to include it.
As a follow-up experiment, we’d recommend exploring the term “contacts” or “email” but also keeping the brand name in play. The term “Contact” appears in 40K terms, and “email” appears in 28K; however, neither term appears in their title tags for the pages where this info would be purchased.