Go to Blog

The Guide to Finding Keywords You Ranked or Lost after a Google Algorithm Update

64
Wow-Score
The Wow-Score shows how engaging a blog post is. It is calculated based on the correlation between users’ active reading time, their scrolling speed and the article’s length.
Learn more

The Guide to Finding Keywords You Ranked or Lost after a Google Algorithm Update

Brett Elliott
The Guide to Finding Keywords You Ranked or Lost after a Google Algorithm Update

Picture this: 

It's 8:30 AM, you are enjoying a hot coffee and open Twitter. You see SEO chatter about a core algorithm update. Curious and wary, you open Google Analytics and traffic has dropped. You frown. You open your keyword tracking tool. Your frown deepens. Everything looks fine in your rank tracking tool. You are not sure which keywords on which pages dropped, and you need to have some answers before your 10:00 AM client call. What to do?

That is what this guide is for. As long as you have Google Analytics, Google Search Console, and SEMrush you can get creative and discover keywords that you lost after the latest update. You may also want to see which keywords are driving increases, and you can use this same process for that as well. 

The first thing we need to do is find out which pages experienced large gains or drops. For this we are going to use the All Pages report in Google Analytics. 

1. Login to Google Analytics
2. Go to Behavior > Site Content > All Pages
3. Set your segment to Organic Traffic
4. Set your date range to only look at the period of time since the algorithm update affected your traffic

If your business is seasonal, then you want to set a comparison in the date range to compare traffic year over year. If you don't have seasonal traffic, then you want to compare it to the previous week. 

Google Analytics all pages report

Notice that we are not focused on sessions or pageviews. We have set our segment to only look at organic traffic, so we are focused on entrances. Entrances is the number of times visitors entered your site through a specified page or set of pages; this means that these specific pages are the ones that experienced a change in rankings, good or bad because the traffic entering that page from search changed. We are not using sessions or pageviews because a person can land on a homepage from organic, then visit other pages on the site and increase organic sessions, even though they did not land on that page from Google. 

Sort your table by organic entrances in descending order so you can see your top performers. Depending on your site, this might mean that you need to change the default 10 rows to something a little larger. You can go up to 5000 rows, but we are mostly looking for the largest drops or gains and many websites get the majority of their traffic from the top 25 or 50 pages on their site. Find the page you want to examine and copy the URL. 

google analytics all pages report organic entrances

I definitely want to know what keywords are driving this sudden large increase in my traffic, so I am headed to SEMrush. Go to the Dashboard and open your website or enter in the search bar. Select Organic Research, then Positions. Scroll down to the bottom, open the advanced filters, and set your filters to match what I have in the image below and hit apply. 

SEMRush filters

This will give you all the keywords within SEMrush's index that your webpage was ranking for, and is a pretty nice list, but we need to export it in Excel because we want to see some before/after rankings. Once it is exported, you will notice that column "last updated" was converted to a timestamp. We need to fix that so select the cell next to the timestamp (on mine this is cell N2) and enter this formula:

=(((M2/60)/60)/24)+DATE(1970,1,1) 

The M2 is referencing the timestamped cell, so if you delete columns, just make sure you update the reference. Right click the highlighted cells, select Format Cells, go to the Number tab, choose Date in the left-hand sidebar, and then choose the date type. Hit OK, and you now have regular dates. 

Add in a name for the column like "Last Checked," then select row 1, click Data in the ribbon up top and set a Filter. You are then going to open up the "Last Checked" filter and deselect all the dates since the update impacted your site. In my case, this removed all keyword rankings that SEMrush collected after August 2nd since my site was not impacted until August 3rd.

Now you have a list of all the keywords within SEMrush's index that you were ranking for before the algorithm update. Copy those keywords and paste into a new tab. Delete all columns except keyword, position, search volume, and date last checked, then name the tab PRE ALGO. These are the keywords you will later enter into a position tracking tool to get your new rankings so you can compare drops or increases from before and after the algorithm update.

Now reopen your filter, and reverse the filter so you are only looking at rankings that SEMrush collected after the update. Copy/paste those into another new tab. Delete all columns except keyword, position, previous position, search volume, and date last checked and name the tab POST ALGO. Because we have the current position, which was checked after the algorithm update, and the previous position as well, we can set a cell formula in a blank cell to =(previous position)-(position) and hit enter; this will show the change in rank for each keyword after the algorithm update*. I like to use conditional formatting to highlight queries where I moved up or down. By doing this, I can instantly see which keywords improved after the update. There are two things I look for - moving up on high search volume queries and moving from page 2 onto page 1. If you are trying to identify big drops, just reverse your logic and highlight the negative numbers. 

excel workbook SEOThis data shows that we moved up from position 4 to position 1 for a high volume keyword after the update, explaining some of the boost.

Because SEMrush does not timestamp the previous position, we do not know for sure that we are comparing a rank change from before the algorithm and after the algorithm for this dataset; this is why it is best to run this analysis quickly after you notice an impact to your traffic - I recommend doing it within a week. 

Okay, this is cool, we now have some of the keywords that increased or decreased for this page. But we are still missing some data. 

We need the other piece of the puzzle. 

Open the PRE ALGO tab and upload those keywords to a keyword tracking tool. Any which one is fine, but I use SEMrush for this because it is fast. You can find the position tracking tool under projects when you set up your domain in there. 

SEMRush position tracking

Once you have your keyword data, export it into a spreadsheet again, then copy it into a new tab on the sheet you have been working in and name it NEW RANKS. Open your PRE ALGO tab and change the name of the "position" column to "previous position." Insert a column next to it and name it "position." Bring your new ranking data over from the NEW RANKS tab using the Vlookup function. Now you have all the keywords for that page from SEMrush's dataset, and you can see where you moved up and where you moved down. 
 

BONUS: 

People who know me know that I am not a huge fan of Search Console for keyword data, but there are times when it can still provide value.  

Follow these steps:

1. Login to your GSC account.
2. Go to the performance report.
3. Set your filter for the date range you want.
4. Set a page filter for the URL you're investigating.

In my case, I am looking into an increase, so I set my date range to begin the day the algorithm impacted my site. If I were investigating a drop, then my end date would be the day before the algo affected my site. It should look something like this:

Google Search Console page level data

Scroll down to see all the queries that drove impressions and clicks to your site. We want this data in a spreadsheet. Change the rows to their maximum limit (500) and export to CSV. 

Note: You may notice as we compare the two that Google Search Console is telling me that page received 876 clicks from search while Google Analytics is telling me it received 1,071 organic entrances in the same time frame. This is why I do not believe this data is complete, but I do believe it is accurate, so we will use it where we can. 

Add a new tab titled GSC in your spreadsheet and copy the Search Console data into there. Use your VLOOKUP function to marry your click data to your keywords from SEMrush. It is easiest to view this next to the rank change column we created, so insert a column and use VLOOKUP to bring over the click data. Now you can see which keywords Google is telling you has been driving clicks to your site, along with data from SEMrush that shows what moved up and what moved down.

You may be wondering why we are going through this with Search Console. It appears that the bigger the search volume, the more clicks you get, and it is all as expected. Except no keyword index is complete and one of the first things I noticed is that the keyword that Google Search Console says is driving the most amount of traffic (126 clicks in 5 days) is not tracked in SEMrush's index. Since that is my largest traffic driver, it is definitely a keyword that I want to keep an eye on and would be adding to my rank tracking tool. 

The important thing to remember is that we are not going to get flawless and complete data from any source, but using this process, we can find many keywords that are driving traffic or stopped driving traffic after the update. 

Upgrade Your Tracking

Take this as an opportunity to promote why you need a bigger budget to track more keywords. It is much easier to look at daily historical data and just know where the increase or decrease occurred while looking at both your pages and your keywords. Map out the keywords that impacted traffic to different pages and use the insight to estimate how much of an increase you will need. When you know how Google altered rankings for your website, it is very easy to examine the SERPs, compare your page, and put a counter plan in place to climb back up the ladder.

Like this post? Follow us on RSS and read more interesting posts:

RSS
Brett Elliott is a full time SEO Analyst at Strategic America and Founder of White Chain SEO, a digital consultancy that provides turnkey SEO solutions and has a two-part mission: evangelize SEO to the uninitiated and advocate for a better industry by teaching white hat methods that work. In his role at Strategic America, Brett focuses most of his efforts on Enterprise Local SEO and manages over three hundred websites for clients in home services. In his spare time, Brett is a devoted father and weekend warrior, with just as many projects at home as he has SEO clients at Strategic America.
Share this post
or

Comments

2000 symbols remain
Shaju Devessy
We have noticed that list of keywords removed from all pages after August google algoritham updates. What should we do next
faisal saleem
I will read it again, good content
This is fantastic. Great approach and I love the details here. Thank you.
-Sunny Kumar
[link removed by moderator]
Moss Clement
Hi Brett, you did a great job here bro. Your article is fascinating with amazing tutorials. I learnt a lot already. I recently wrote and published an article on SEO Guide where I also talked about finding keywords you're already ranking for. In my post, I illustrated the use of Google Analytics and GSC to find keywords you're already ranking for. However, the concept of finding keywords your lost keywords after Google algorithm updates is a clear cut hands down win.

Good job👌👍!
Anthony Godley
Hi Brett,

Great read.

I thought I would write an SEO Guide to help people understand how SEO has evolved and the ranking factors that now exist in the modern SEO age including UX consideration and how UX design is an important factor when it comes to successful search.

http://theseoteam.co.uk/complete-seo-guide/

I hope this is ok to post here and actually I am very hopeful the followers will find this guide very useful :)
Thank you very much Sir for share this more informative post. You really explained good about everything, i was confused about those keywords who were ranking before the update but after the update they are not visible up to 100 results. Now i will apply above process to get rank again. Thank again
Hi Brett,
Organic clicks are not organic entrances. In fact entrances aren't even exactly the same as sessions, they're all close, but they're not the same. The subtle differences between each are where and why there a differences in their counts.

> Organic clicks are associated with an on click event from a search result.
> Organic sessions could be an organic click, or they could also be a return visit that has first click attribution set by an organic click. Therefore even if it was a return visit from something other than an organic click, it would still be an organic session.
> Organic entrances fire if the first hit from analytics is a pageview event, hypothetically if the first hit from the session is an event, then it won't register the entrance.

You can rely on the organic click totals in Google Webmaster Tools / Google Search Console, you just can't rely on specifically which query that click came from.
Anand Joshi
Thanks for this Informative and useful post.
Xavier Clapin-Pépin
Great, I followed step by step your tutorial and it works well!

I am wondering though if all that work is worth it. Once you identified your top keywords, why don't you simply use SEMrush position tracking (tab "Landing Pages") to compare the results before and after the algo update?
Brett Elliott
Xavier Clapin-Pépin
Hi Xavier, good question. I honestly love that tool. But to the best of my knowledge it will only show information from the date you start tracking, so if you were not tracking the right keywords before the update it will not be much help. Someone from the SEMRush team jump in here if I'm wrong about that.
Xavier Clapin-Pépin
Brett Elliott
Hi Brett,

I agree with you in the situation that you do not monitor all your target/potential keywords before the Google Updates. It can be because you have many clients/projects on SEMrush and you reached your keywords limits and decide to not monitor all target/potential keywords.

Nevertheless, if you're serious with your SEO development for a client, first thing to do is a keyword research to start monitoring most of the target/potential keywords. Also, at least once a week we edit the keywords we monitor with different techniques (add or remove keywords).

It's challenging to keep an up to date position tracking list but it's so important and if you do that then it's fast to understand the changes when a Google Update happens.

It's so important that for our clients for Local SEO we have to pay for BrightLocal for their local SEO tools because SEMrush is not giving enough data for local position tracking for instance a local based keyword has different ranking positions for Google Search, Google Maps, Yahoo, Bing, Bing Local.

Brett the way you wrote your article was amazing, easy to follow the instructions, hope you will post more articles.

Cheers
Melanie Nathan
Xavier Clapin-Pépin
Agreed that it’s a little too much work to find out what keywords you gained or lost. You can actually find that info in two clicks with SEMrush. Still a useful guide though 👍🏻
Christos Shiatis
Cool! That's a very helpful article, I like how you combine different tools to get as more accurate data as possible!
Sanaullah Kiani
Thank you very much for the update help.
ILYA ABINOV
Thank you for this guide!
C. Alex Velazquez
Holy cow Brett. I've been doing this a long time, and this post still taught me at least three new things. Great content, exceptional 'how-to' walk through using various tools we all spend our days inside of. Another winner! Thanks man! Cheers. (keep them coming!)
Brett Elliott
C. Alex Velazquez
Thanks, Alex! It was a blast to write!
Very good article ,will follow and study.
Am sorry ,may i ask if we can do this SEO without any code knowledge.?
Brett Elliott
Kyle Lee
Hi Kyle, it doesn't require any programming knowledge to follow the process detailed above. Good luck with your efforts!
Brett Elliott
Thank you so much for the response.
Brett Elliott
Not sure may i ask one question, i am confused with the keywords related to the product. for example ,i have one t-shirt ,totally we can describe it as long sleeve flower print pocket t-shirt, so my problem is that what other keywords i could use for this t-shirt ? i mean the t-shirt only has such features. if i use other words ,it will not point to this product, it will be a problem ?would you please to help me out ?thank you
Melanie Nathan
Kyle Lee
I can’t help but wonder how many people search for “tee shirts”.