SEO Experiments That Will Blow Your Mind

Larry Kim

Nov 23, 20205 min read
SEO Experiments That Will Blow Your Mind

In SEO, everyone has their opinions, but what I love about SEO is that at the end of the day, data wins the argument; either it worked, or it didn’t. That is why Will Critchlow of Distilled, and I got together to talk about some SEO experiments, hopefully, some surprising ones, and some you may have tried yourself. Some of these crazy experiments will blow your mind.

In our recent webinar, Will and I covered ten experiments — here, I will cover six.

I want to start with some examples of really common advice that you probably have seen on many blogs, if you have ever hired an SEO consultant, or if you ever worked in SEO, you have probably given this advice. There are also some made-up examples where the clients and customers aren’t real. These are equivalents of tests that we have actually run with some anonymized websites:

In order to measure this, I essentially use “time-lapse photography”. In this example, I was tracking 1,000 different keywords across all of last year, the same 1,000 keywords and I took snapshots in May, June, September, etc.


I found that the order of these keywords impacted the search rankings. In this study, the search rankings were changing from month to month in a very predictable pattern. The keywords that had higher click-through rates were floating to the top of the page, and the keywords that had lower expected click-through rates were falling to the bottom of the page.

Second SEO Experiment: Does click-through rate actually impact the rankings?

It is obvious that rankings impact click-through rates, but is there a feedback loop? Is the opposite true? Does a higher than expected click-through rate actually feed back into the algorithm somehow and result in a higher position?


We looked at relative organic click-through rates to the extent in which every keyword was beating or being beaten by the expected click-through rate. What we saw was that all you need to do is beat the average expected click-through rate for any given spot for 3%, on average, to be potentially be promoted into the next position.


Third SEO Experiment: Title Tag Optimization

In this example, the original title is the red one on the top below on the slide where it says "Guerilla Marketing: 20+ Examples & Strategies to Stand Out," which is the old way of doing it where you take the primary keyword and stuff it at the beginning of the title page. However, in order to make it more “clicky,” I rearranged the words to make it " 20+ Jaw-Dropping Guerilla Marketing Examples and Strategies." By adding that action word "jaw-dropping" and then moving the number to the front, it had a profound impact on both click-through rate and rank.


Click-through rate optimization is interesting because even if you don't change rank, you are going to double your traffic because you are going to get more than your fair share of clicks for your given position. But, there are ways to increase your click-through rates without even changing your position, and if it does change position, which is often the case, it is almost like a double bonus.

CTR boosts are very, very valuable. Even if you don't believe my theory about CTR improving rankings, it is valuable because higher click-through rates mean basically more and more clicks.

Fourth SEO experiment: Do engagement rates impact rankings?

We know that Google measures dwell time because they used to have a feature where if you hit “back” very quickly you could actually block the site, and right now, it is the same thing on mobile - if you hit “back” quickly, it will say “People also search for.”


Google is measuring the time that it takes for you to click on something and return to the same page. Could they be using that information to say the search results are a sign of relevancy or non-relevancy? Sure enough, I was unable to measure the dwell time because that’s measured by the Google search page, and we are unable to measure other equivalent metrics such as bounce rates and time on site. If you have a very low bounce rate, then you are eligible to show in the top four positions. Conversely, if you have a very high bounce rate, you become less likely to occupy those top positions.


The discontinuity in the graph above suggests some kind of an algorithmic filter (as opposed to a naturally occurring relationship). If you have a decent time on site, you seem to be eligible to show up in the first six positions. Otherwise, you are relegated to position 7, 8, 9, 10, etc.


Fifth SEO Experiment: What’s the real relationship between social sharing and organic rankings?

For many years, Google denied that they use any social signals to rank pages, but if you have ever done any SEO in your life, you have probably noticed that your best SEO is rocking it in Facebook shares, too. There is a correlation, but it is not what you think, they are not counting the shares; instead, the search listings that people like to click on and have high click-through-rates also have very high click-through rates when shared on social media channels.

This is because the same emotions that make people want to click on things are the same emotions that make people want to share things on Facebook:


This graph shows click-through rates normalized by the position on the search results page, versus post-engagement rates for Facebook posts. As you can see, the content that does really well tends to do well on both, because both Google search and the Facebook news feed algorithm employ machine-learning systems that reward high user engagement with greater visibility.

Sixth SEO Experiment: How does time (bounce rate) impact SEO rankings?

In Google Analytics, you have a comparison report that tells you the time spent on page. This report shows you all of the pieces of content that are driving traffic to your site. In the below example, the timeframe used is 2015 (January to December of 2015). You can see the top pages for SEO. Most of them had above average time on page (those are the green bars), but, 1/3 of them had red bars, which means they were actually under the average time on page for the site average.


Let’s fast forward (October 2016 to January 2017); you can see that it has changed dramatically and that Google has found and eliminated all of the donkey pages. All of those pages that had below-average time on site, Google has somehow managed to find them and eliminate them from contention among my top evergreen SEO pages. They have no business ranking on any of those keywords; they are vulnerable pages that either need to be fixed or they are soon going to disappear.


These are just a few of the experiments covered in my recent webinar. Some of the experiments mentioned above are similar to SEMrush’s recent Ranking Factors Study. While we have observed slightly different things, I believe we are looking at the same phenomenon from a different point of view. If you would like a more thorough explanation or to see many of the others completed, you can view them here.

If you would like to view my slides from the webinar, you can link to them here. You can also access Will’s by clicking here.
Note: These SEO experiments were completed in 2017.

Author Photo
    In 2007, Larry founded WordStream, an innovative search marketing platform that enables even novice marketers to be successful at paid search. Today, WordStream is a multi-million-dollar company with thousands of customers worldwide and has earned industry accolades for both innovation and growth.     Larry founded MobileMonkey in June 2017 along with a team of former WordStream Executives and still serves on WordStream's board of directors.     He is widely considered one of the most influential experts and thought leaders in Digital Marketing today and was voted the most influential PPC expert in 2017, 2015, 2014 and 2013 according to the PPC Hero Blog. He has earned multiple industry awards and is frequently a featured speaker at industry events. He is a active on social media, a popular blogger, and speaker.