Vintage Clothing Brand Increases Organic Traffic by 21% with SEO Split Testing

Business Challenge: measure the impact of SEO efforts for eCommerce during unstable demand

Exclusive Concepts has been working on digital marketing for Unique Vintage since 2013. The apparel retailer is known for its dresses and special occasion clothes  - a category that saw a significant decline in demand during the COVID-19 pandemic. 

Following the store’s natural decrease in organic traffic in 2020, Exclusive Concepts was striving to gain traffic back with optimization for relevant categories that had a better chance of recovering during the pandemic. 

“While applying various SEO strategies, we faced a dilemma: how would we know if the traffic changes were caused by our actions or a change in demand?” explains Rebecca Stewart, Director of Organic Search at Exclusive Concepts. “Black Friday and holiday season were nearing and we also had to assume there would be changes in customer behavior due to COVID-19.”

The agency decided to test their solutions with SplitSignal - the new SEO A/B testing tool powered by Semrush.

“The direct impact of SEO efforts are very difficult to quantify - we can test pretty much everything else other than SEO. Testing our SEO optimization tactics similar to how we test CRO has been something we’ve always wanted to try.”
 

Solution: SEO A/B testing

Test Hypothesis

“We had a theory that Unique Vintage customers love to see when new styles have been added, so we added the phrase ‘New Styles Just Added!’ to the beginning of our meta description,” says Rebecca. 

Here is an example of how this would look in a mock-up of SERP:

Selecting control and variant pages (bucketing)

Exclusive Concepts chose the site’s Collections section (consisting of 28 categories) as the test location.

The variant and control groups were designed to be statistically similar to each other: both included pages from all 28 categories, with each page getting a similar amount of traffic. As a result, the test was grouped into two sets of pages: control group (136) and variant group (137). 

“New Styles Just Added!” would be applied to the variant group only. As a part of the test, two different templates would be live at the same time, so there was only one version of each category page (read more about the difference between CRO A/B testing and SEO A/B split testing).

Running the Test

For the test to be accurate, SplitSignal requires access to Google Search Console in order to analyze 100+ days of historical organic traffic data. This information is used for the estimation of statistical significance of the proposed change once the test is launched (learn more about how the SEO A/B split testing works).

The graph below shows the correlation between the control and variant groups in the previous 100 days, as well as the test period. The accuracy of the model is dependent on the ‘real’ traffic and the ‘predicted’ traffic matching as closely as possible during the historical time period (pre-test-launch), and the ‘real traffic’ line being consistently higher than the ‘predicted’ traffic line post test launch.

Setting up the test took 1 hour; the test was launched October 15th, 2020 and ran for 28 days (until November 12).

After 28 days, the test showed a significant improvement in the variant group performance: the pages with “New Styles Just Added!” in the meta description received 21% more organic sessions. 

“I was pleasantly surprised that a change to the meta description had such a noticeable effect on traffic,” shares Rebecca. “While meta descriptions aren’t a ranking factor from a keyword relevancy standpoint, this test proved their importance in driving more users through a higher click-through rate.  That +21% increase in sessions could amount to a +5% increase in sitewide revenue over 12 months.” 
 

Results:

+21% traffic growth on variant pages

Sitewide implementation of the variant meta description would increase the sitewide revenue by 5% in the next 12 months.

Bringing the measurability of PPC & Ads Channels to Earned & Organic

Exclusive Concepts can now easily present quantifiable results of their efforts.

Proving the ROI makes upselling easy for agencies

SplitSignal helps Exclusive Concepts estimate the revenue impact of the test, making it easy to justify the costs of running SEO A/B testing. Unique Vintage is excited to invest in a service that delivers revenue growth.