The means and inclination to test SEO changes is growing within the industry (finally), but a willingness to test does not always mean it will be successful and worth doing.
Not everything is worth testing, and not everything is easily testable. So what follows are some of my lessons/examples from testing SEO changes (split testing or otherwise).Â
What to Test
How do we decide what to test? Selecting the right test is not always as simple as simply âwhich will have the biggest impactâ here are some considerations when choosing a test:
- Speed of changeâHow quick/easy is it to make the change? Even if you can modify the page with test tooling, do you have what you need (logic/information) to make the change?Â
- Who else within the business needs to sign it off? If you are significantly impacting the experience of users and will need the attention from other teamsâUX/Ecommerce teams, for exampleâyou seldom can run these tests without getting buy-in first.
- Avoid clashing with UX testingâIf other âconventionalâ UX tests are being run, you need to ensure that you donât make SEO changes while another testing is happening. The two tests may clash and rule out any validity in the results.
- The impact of the changeâDo you think the test is worth running? Sometimes, the lightweight tests arenât worth testing.
- Is Google likely to crawl/index the page?âIf you have a large/complicated site or JavaScript is required to render key content, it may take Google too long to index the test bucket. You want your test to be complete in 4-6 weeks (max). If Google doesnât pick up the changes within two weeks, it may invalidate the results.
With the above in mind, letâs move on to the examples.

This list isnât exhaustive, the brilliant thing about testing is every website, niche, and tech setup will provide opportunities to test in ways that will surprise and shock you.
*The team required to make the technical change is an assumption weâll have for everything here.
Insights on Different SEO Change/Test Types
Tests and changes will always deliver different results depending on the site and circumstances around the test itself. In some situations, what benefits one site may actually detriment another. This means applying learnings from some tests to other sites is not always wise.
Even with this in mind, some aspects of testing and the experience of testing are worth sharing, even if it helps develop your own thinking and approach for doing this yourself.
Title/Meta Descriptions Can be Fertile Ground
If your titles/descriptions arenât especially good, but you donât have the resources to change them all, a split test is a great opportunity here.Â
This is a relatively quick and easy test to apply (with the right logic available), and it is easy to see on the SERPs if Google has âseenâ the test.
Trying to Change CTR Can Be Frustrating
Changing CTR can be challenging because you assume it will deliver a net benefit to the userâs experience on the SERP. As you often make these changes at scale across the test group, some may see improvements, but others may worsen things.
A classic example is adding review ratings via schema, or a title change can display some products/services which people rate well. Still, anything with a poor review will also get greater attention. Putting bad reviews in SERPs doesnât often help clicks!
Mark-Up Changes Can Be Quick & Easy
It can be relatively easy to make mark-up changes. This can be very useful to justify making changes to page templates to roll out âbest practicesâ in headings or Schema. Maybe it can be used to help justify an altogether more âradicalâ change without fully rolling it out.
While a markup change can be a good test, the results arenât always significant.
Small changes often equal minor improvements, so youâll need to factor this into your decision-making.
You Can Hypothesize Losses too
Depending on the testing circumstances, it is possible to predict a decline in performance based on change. If another team wants to make a change, youâre not confident with it. You can insist on testing to validate a hypothesis that it is a bad idea.Â
Testing Site-Wide Elements
Testing site-wide elements (for example, main menu or footer) is highly problematic as an A/B Test. We need to ensure we are serving Google two roughly-equal page groups (in traffic size & user behavior) as a test & controlâbut this isnât possible with a persistent element. A test where the menu is only different in some cases, and not others will be bad for search engines and users.
The primary workaround here would be to do a time-based test in this instanceâso make the change and then run causal impact analysis on the data after. You lose the advantages of an A/B test hereâbut the alternatives arenât strong enough to provide trustworthy conclusions.
Time to Get Testing
Once you have a strong hypothesis, buy-in from the appropriate stakeholders, the ability to test/measure the results, and the means to serve the test itself (if youâre A/B testing), you are in the prime position to get some great insights and data. You can always start testing with SplitSignal, a Semrush tool for SEO split testing.