Before you start: if you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we’re suggesting you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
This time the majority of our followers were WRONG! Read further to find out why.
The Case Study
The value of SEO content on ecommerce category pages, whether at the top or bottom, has been the subject of much debate. These pieces of content are often created primarily for SEO purposes, to increase visibility for relevant keywords, rather than to enhance the user experience. Google’s own John Mueller has commented that these texts are sometimes seen as attempts at keyword stuffing, which can negatively impact the page’s overall view. While some content can be helpful, he warned against overdoing it with large article-style content below product listings, as it can make it harder for Google to understand the page’s intent. Google also stated that there may not be a significant difference in terms of SEO between placing the content at the top or bottom of the page but that a well-crafted piece of content can help users better understand the category and its offerings.
Naturally, these discussions and remarks from Google have piqued our interest. Creating SEO content for hundreds of thousands of category pages can be a significant investment in terms of time and resources. Therefore, it is crucial to understand the true value of this type of content. These insights have prompted us to conduct a series of split tests to examine the impact of different approaches to SEO content on ecommerce category pages.
We recently conducted a split test for one of our major ecommerce clients, where we removed the SEO content from the top of their category pages to evaluate its impact on organic performance and search engine visibility.
The Hypothesis
The website in question wanted to test if removing their SEO text, about 100 to 200 words at the top of their category listing pages, would have a significant impact on organic traffic.
Internally, our client had extensive discussions about the value of the SEO content and hypothesized that removing it would improve their SEO performance. They believed that this change would allow their category pages to better focus on providing users with the product listings they came to the page for, putting more emphasis on search intent. Additionally, they expected that Google’s algorithms would be better able to understand the intent of the page and, in return, could possibly improve its visibility in search results.
The Test
Our client sought to assess the impact of removing SEO content from the top of their category pages, and they used the SplitSignal tool to conduct the test. They selected a sample of category listing pages as either control or variant.
The split test ran for 30 days, during which Googlebot crawled and indexed around 97% of the pages.
The Results
After 30 days of testing, we found that removing SEO content from the top of category pages did not have a significant effect on organic traffic at the 95% confidence interval. The cumulative view of the test showed that the blue shaded area did not perform below or above the x=0 axis, indicating that the test was not statistically significant at the 95% level. As a result, we cannot confidently attribute any increase or decrease in traffic solely to the change in SEO content.
Based on these findings, we conclude that removing SEO content from the top of category listing pages does not have a significant impact on SEO performance.
Note that we are not comparing the actual control group pages to our tested pages. But rather a forecast based on historical data. The model predicts the counterfactual response that would have occurred had no intervention taken place. We compare this with the actual data. We use control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and consider it. By filtering these external factors, we gain insight into the true impact of an SEO change.
Why
Based on the results of the split test, we can conclude that the small piece of SEO text removed from the top of the ecommerce category pages did not have a significant impact on organic traffic for our client. The hypothesis that removing the SEO content could improve SEO performance and help Google better understand the intent of the page did not hold up under testing.
Furthermore, the text itself seemed to have no SEO value, as its removal had no discernible effect on SEO. This indicates that the content was not contributing meaningfully to the page’s overall SEO performance.
Additionally, we conducted an analysis on organic impressions using the SEO A/B test analyzer to evaluate if there was any impact on visibility in Google search results. The results of this analysis showed that the removal of the SEO content did not significantly impact organic impressions compared to our modeled control group.
By analyzing both organic clicks and impressions, we can confidently conclude that removing the SEO content from the top of ecommerce category pages did not have a significant impact on SEO performance or visibility in Google search results.
In summary, our findings suggest that this particular ecommerce business may not need to invest significant resources in creating and optimizing SEO content for their category pages, especially if the content provides limited value to users. Instead, the business can focus on creating high-quality product listings and ensuring that their category pages are well-organized and easy to navigate.
We are excited to continue challenging Google’s statements about the value of SEO content on ecommerce category pages through our ongoing split testing efforts. We have been conducting tests on removing larger pieces of content placed below the product listings. These results will be shared in future case studies, providing further insight into the true impact of SEO content on ecommerce category pages.
There’s no one-size-fits-all solution in SEO. What works for one website may not work for another. So, if you want to succeed, test what works for you!