Submit post
Go to Blog

New SEMrush Data: Fresh Keywords, Precise Volumes

67
Wow-Score
The Wow-Score shows how engaging a blog post is. It is calculated based on the correlation between users’ active reading time, their scrolling speed and the article’s length.
Learn more

New SEMrush Data: Fresh Keywords, Precise Volumes

This post is in English
Elena Terenteva
This post is in English
New SEMrush Data: Fresh Keywords, Precise Volumes

This year SEMrush celebrates its 10th anniversary! So let’s kick off the celebration with a big announcement and a sneak peek at a little something we like to call SEMrush 2.0.

Last year we were recognized as the Best SEO Software Suite of 2017 by the European Search Awards and by both the United States and the United Kingdom Search Awards. But perfection knows no limits.

With our upcoming innovations, we will keep raising the bar for digital marketing toolkits. And since the cornerstone of any analytical process is data, that is exactly where we are starting our renovation.

Data: What Makes It Accurate

Before we dive into detail, let's recap the main ideas and arguments concerning keyword search volumes and why they are important for digital marketers.

Volume indicates keywords’ popularity through the number of searches and is a very valuable metric for SEOs. Here it gets interesting: how do we get this data?

Google is the only official source of keyword search volumes, but over the years it grew less and less enthusiastic about sharing the information. And this is where clickstream data comes into play.

Clickstream is an anonymous record of users’ activities in web browsers and on various apps. This information is aggregated by data-providing companies and sold to all kinds of analytical platforms. Collecting this information is the easy part, processing it to get valuable results is much trickier. False volume estimation leads to faulty keyword targeting, which eventually ruins your SEO and PPC campaigns.

To ensure that you get the most relevant and precise data, SEMrush is implementing a new machine learning algorithm for search volume forecasting and validation. And we can claim to have the most accurate volume estimations on the market. Now let's see how we have achieved it.

New Data: Keeping Up With The Changes

Search is ever-changing: new popular things get the spotlight while others perish, the demand mutates, and users cultivate new habits. To stay current we have to be flexible and to be flexible we needed a change in the core of our analytical framework – the keyword database.

The first innovation affects the keywords themselves.

We are retiring old and untrendy queries and filling our database with the finest hot-off-the-press terms. Tired of shoveling out piles of worthless keywords? We are relieving you of this chore by providing only the contemporary keywords that have significance for SEO. Reinforcing our status as the best solution for digital marketers, we will deliver the most profitable traffic-driving terms and high-quality longtails.

The second change concerns volumes and the aforementioned clever machine learning method.

We will keep updating our volumes monthly, but with the new algorithm and architecture, we can raise the standards for estimation accuracy even higher.

But, like many great things, our improvement has one small drawback. A new database means new digits, which may affect your reporting. The total number of keywords can swing, and you might notice some spikes and drops in traffic estimation. We hope it will not become a major fly in the ointment because eventually, you will get the best tool with the most accurate data.

If you are concerned about the reduced number of keywords, don’t be. Our database will stay just as large, but now it will contain the most adequate keywords for website optimisation.

You will still have our Keyword Magic Tool with all of the fantastic keyword research features at your disposal, if you are not going after the keyword analysis for the specific domains, and need a full database for brainstorming and diversifying your semantic core. This data will also be regularly updated with the new keywords and freshest metrics.

Data 2.0: The SEMrush Way

Now let’s enter spoiler territory with a promised teaser of our new Domain Analytics. To provide the best experience we are sprucing up the interface and augmenting our reports with some remarkable features. Plus, the new architecture will immensely speed up our tool.

To optimise the SEO workflow, we streamlined our Overview Report. It will show only organic metrics with clear-cut widgets for traffic estimation, keywords, position changes, SERP features, competitors, top pages and top subdomains.

Competitors are assessed in a brand new way that is unique to the market. Managing competition research has always been our strong suit, and yet we are adding a new level of precision to the analysis. Now it is not only about the common keywords but also about rivals’ proximity in the search results. By considering the closeness of positions, SEMrush excludes contenders from the outskirts of the SERPs and surrenders to you the real opponents.

Traffic is now presented with absolute values to improve the experience of our users. Be aware that these are still estimations based on volumes and average CTRs. While the numbers are not exact, they are close to the real picture and accurate in terms of correlation. So, you should not perceive this estimation as the ultimate data source, but rather as a strong asset for comparison and analysis.

Positions and Position Changes reports have a brilliant addition. Now, all filtering that you apply will not only affect the data table but will also instantly change the graphs. This is unprecedented for such a huge database. Having a visual representation on the fly will immeasurably enhance your analytical process.

SEMrush Domain Analytics instant graphs

Our advanced filtering with instant graphs creates an incredible sandbox for nuanced trend analysis. Evaluate in a multitude of ways how the website is performing over time, assess specific subdomains and pages, and filter them out by various SEO metrics and keywords of different types.

And that’s not all, folks. More great improvements are on the way, so stay tuned.

Successful SEO requires multiple interrelated activities on all fronts: competition, keywords, link building, on-page and technical optimization. It creates a need for diverse tools, which is expensive. SEMrush solves this problem with an award-winning all-in-one toolkit that includes 17 tools covering all SEO fields. We’ll walk you through your SEO workflow, explaining how to get the most out of our toolkit.

Get Free PDF

Like this post? Follow us on RSS and read more interesting posts:

RSS
Elena Terenteva, Product Marketing Manager at SEMrush. Elena has eight years public relations and journalism experience, working as a broadcasting journalist, PR/Content manager for IT and finance companies.
Bookworm, poker player, good swimmer.
Share this post
or

Comments

2000 symbols remain
¿Existen libros traducidos a Español?
Hi Elena! I have a question about the new keyword tool: if I make an analysis of a website I've been working on since December 2017, I notice that the number of keywords on the first page of Google has notably increased. How can I be sure that this data is reliable and that it has not been affected by the tool upgrade? In other words, can I claim - without any doubts - that the increase in the number of keywords has been due to my work and not to the tool update? Thanks!
SEMRush's organic traffic reports are a mess since their last keyword update. Every site is pretty much staying the same (gaining and losing the same amount of keywords). The tool was much more accurate before their keyword update. I'll be canceling my subscription. They ruined a good tool.
Yea the keywords features which we used this service for dropped below ahrefs in quality, so we upgraded over there and use their listings instead.
Dan
I agree - I'll be switching to Ahrefs too. SEMRush made a huge mistake with their keyword update. Completely screwed up every site's estimated organic traffic.
Carl Nickles
Yea I gave them a couple months, but I found myself using the service less and less, couldn't justify the cost over the competition like SERPchecker and serpbook etc.
Elena Terenteva
Dan
Hey Dan! Hey Carl! I’m very sorry to hear that the new update didn’t impress you. User feedback is one of the things we cherish the most, and I’d really appreciate if you could find some time to discuss it with us. Ruining a good tool is definitely not our goal :) Wanna keep you on board!
The Keyword volumes have gone mad. It looks like google PPC data when a set of keywords are grouped together in a keyword bucket and all attributed the same volume. Really not an improvement!
Elena Terenteva
Neil Johnstone
Hi Niel! Following the global trends, we have decided on combining keyword variations into one main keyword in our Domain & Keyword Analytics as part of this update. Some misspellings and variations would thus be combined into a single version based on popularity, with their volumes counted as a whole as well.
This step was taken as part of our global databases update in order to provide you with the most relevant keywords and their metrics for your research. Some misspellings and keyword variations may still be encountered in our databases, but overall the combination approach has been chosen.
If you would like to monitor specific keywords that are not currently found in our database our Position Tracking tool would be the place to go! You can find more details on the tool here: https://www.semrush.com/kb/32-position-tracking
Hope that information was helpful! If you have any other questions — feel free to reach out to us!
Elena Terenteva
This is a shame, as it ruins the ability to analyse the data and choose what keywords to optimse a page for. This is the entire reason for using SEMrush. With the obvious misspellings this is easy to ignore, but some keywords are very similar but have huge differences in volume. Sadly with this update SEMrush is far less useful. I would urge you to reconsider your approach.
welp, semrush is not doing too hot looks like a good time to swap over to moz till things settle down
Dan
Moz is volatile as well. DA metrics is fluctuating a lot in the last few months.
Karan Sharma
yea I've noticed that too
Elena Terenteva
Dan
Hi Dan! My apologies for taking a while to answer. I can confirm that our global databases update has finished at the end of March — are you seeing any unstable results in any of the Domain & Keyword Analytics reports? If so, could you kindly get in touch with us via [email protected] with some examples so that we could review the case? That would help us improve our platform for you!
Sooo... it's been 3 months and the keyword count is crazy volatile. Any update on the end of the process?
P.S. It would be super helpful if you told us the total number of tracked keywords along with the stats (separately for every single day, of source).
when will the process of updating the keyword database end?
Since the new update I must say the numbers shown are very inaccurate, even certain keywords are way off rank wise (showing better than they are) and others have not updated in 2 months+ ......
Hi, its true that will be another update and another month more? We had suffer a month with lost of 50% keywords data in all the sites we monitoring, and not little volumen words, some of them even with volumes of 450k or more. We pay the service but cant have reliable data and will be the same the next month? I want to ask if will be some compensation, we are paying the full monthly fee but with the half of data, and becose incomplete data we have to pay also another ranks services for the customer reports.
Jose
My sale id is 405720, thanks.
Elena Terenteva
Jose
Hi Jose! 50% is a serious drop and it has to be properly examined - so customer success team is working on your case right now and I believe they will answer you shortly. From my side, I can tell that we value every our customer and we'll do our best to see you among them :)
For one of my other site [link removed by moderator], there is a huge gain reported by SEMrush (https://www.semrush.com/in/info/poultrybazaar.net) but Google Analytics reports a 20% (+) drop in traffic.
I'm afraid the new update isn't working at all for me. My sites are not huge ( less than 60000 visits per month) , SEM has always undershown the figures google analytics shows me for search traffic by more than 50%. I have lived with this because using SEM in combination with other tools it still formed a useful role. However now I see that SEM is undershowing reality by something like 72% one has to wonder if it has any utility at all.
I used it particularly to help me to research keywords but if the volume numbers are so far adrift of reality it simply doesn't appear to be reliable anymore.
I called SEM regarding this and their comments were that their algorithm was different to Google ( this is obvious but the figures are represented as giving an insight to google search volumes when actually they appear to do no such thing), they also said things would settle by the end of FEB ( personally I cannot see the new algorithm suddenly reversing where it has been getting to and then going further).
It's a very great shame because while all the systems have their faults I had preferred SEM's to others but now that it is so far adrift of reality it seems pointless to base any decisions on the data it supplies. .... I wish it were otherwise.
I should say that I have been a customer of SEM since 2013 and have also taken the time to write to them sharing the detail of my analytics data with them and raising the problem in a factual way but have yet to get any response.
As I said it's a sad turn of events. My own view is that they set out to make things better but have somehow made a huge error meaning they have taken a very significant backwards step. What a pity . I'll keep checking in and perhaps come back and use it again if they find some way of fixing it, in fact I'd be thrilled if they did manage to fix things
Elena Terenteva
antt
Hey! Thank you for your feedback - it's quite important. I guess if you could provide us with an examples of the keywords, which were lost for your website, we could understand what might went wrong. Pretty sure support guys will answer you soon - they always do :) Thanks for your comment, please come with any questions.
Elena Terenteva
Thanks Elena. I don't believe it's a matter of small keywords going from the estimation as there remain pages and pages of small volume keywords there ( 10 searches a month- your estimate of total searches for that term). I think the core issue is that estimates for volumes by keyword or estimates for click rates by position have been out before and have simply been made much, much worse by the update. Historically you were listing us as performing against 5k keywords ( and your search numbers were 50% out) it's now just over 3K keywords ( and the search numbers are 72% out), crucially your estimate of keywords in positions 1-3 has dropped by 50%. As I said compared to the traffic from google analytics, as I write this, the SEM numbers are 72% astray. I have tried to share my analytics numbers with your guys but I fear nothing will happen.

On another point another of my sites which launched six months ago and last month grew 320% ( from a small beginning) and this month is showing week on week growth of 14% ( analytics) is also way askew...... SEM picked up the growth spurt last month ( but only half represented it), yet it is currently showing volume decline rather than continued growth....... I would at least expect the direction of travel to be correctly identified even if the volumes were adrift. Your tech guys are also aware of this.
As I said let's hope some form of utility can be brought back to the results as I used to be dedicated to SEM and I fear now the figures have lost all reliability.
I got 12000% growth in traffic. Is this genuine or is the tool going under any testing? Here is the link: https://www.semrush.com/info/premiumsoftwares.com
Hey Elena,

Great post. Loved the new updates and my best wishes to SEM Rush team☺️👍
Heyo,

It looks like this update is affecting CANADAIN domain overview data. It's telling me that my websites have lost ~20% of keywords ranking, but the position tracking reports indicate differently - as does Google Webmaster Tools and Google Analytics data.
Elena Terenteva
Jonathan Berthold
Hey Eitan! Yes, this might happen for some low-volumed keywords - you can still see them in Position Tracking, but they just were dropped from the database. And this update is rolling for all databases - Canadian is included. Please wait a bit more - we are almost done. Cheers
the update will never end?
Elena Terenteva
Almy Froes
Hi Almy! Will end for sure! All graphs should stabilize till the end of the February (for US - end of the February, for other databases - sooner).
When will finish the update? Thanks.
Did the update happen on the 15th or just start on the 15th? If it just started on the 15th how long do you anticipate before the update is done. We use your keyword rankings as one measurement for SEO and it's hard to judge if it's something Google or SEMrush is going doing without knowing when SEMrush update is complete. TIA
Tracy Herbolsheimer
I was told through twitter this update will last around one month.
https://twitter.com/LenaTerentyeva/status/955746691885432832
Hi, Its a week ago but in all sites of my industry I still see for all of them keywords loss of 50-60%. When will finish the update? Thanks.
Jose
I second that, most domains tracked show a loss of keywords.
Great post, thanks. Question: when will the process of updating the keyword database end (or stabilize to some extent)? I'm using the "number of keywords my site is ranking for" report as an indicator of how things are going - if numbers are growing fast, we're doing good. If they're falling slow, it's meh... if they're falling, something's wrong and we need to act. So far, within the last 20 or so months, we fell once for a few days in August last year (and indeed, there were two problems with the site - we fixed them and started growing again) and the 2nd time... yesterday. I'm hoping this is due to the database overhaul, but I'd love to know:
* are you dropping some keywords from the database? If yes, how significant the drop is (percentage of the whole DB would be great).
* how long will it take before the DB stabilizes and doesn't change much anymore.

Side note: I've always missed one single thing from the report - a total number of keywords - I was never sure if our "growth" was due to us REALLY doing better or was it because you introduced some new keywords that we rank for.
Elena Terenteva
Mateusz Mucha
Hi Mateusz! Sorry for a late reply. All graphs should stabilize till the end of the February for sure (for US - end of the February, for other databases - sooner). Yes, some keywords are dropping, but the volume of the database is remaining the same, because we are adding new keywords and I can't tell you the percentage - it's different for different databases. Regarding your case - the drop might be significant because you rank for a lot of low-volume keywords, so to get a clearer picture on the success of your local online performance we recommend to set up Position Tracking for the particular city. If lost keywords were not low-volume keywords - let me know, we'd like to take a look. Cheers
At what time today does this update get pushed? Thanks.
Elena Terenteva
Carl Nickles
Hi Carl! it started on January 15 and will be rolling for a month or so
Martin Woods
The update instant update to graphs is a brilliant feature, I look forward to trying it out!
Elena Terenteva
Martin Woods
Hi Martin! Please give us your feedback later, looking forward for it.
David Iwanow
Wonderful news :)
Elena Terenteva
David Iwanow
Hi David! Indeed, the update is starting today - let us know what do you think.
Darvin Reyes
This is a surprising information, thank you ms, Elena!
Elena Terenteva
Darvin Reyes
You are very welcome, Darvin! We'l continue to surprise you this year :)
Thank you for the informative article, Elena! Over what period should we expect these changes to occur - and when (roughly) can we expect the full changes to be complete? Thanks!
Elena Terenteva
Alex Sturwold
Hi Alex! The update of our largest databases is starting on January 15, today we’ll provide additional info about it. And this will take up to one month to roll out. Smaller databases are already being rolled out.
Oleg mak
Cool news, thanks for your work
Elena Terenteva
Oleg mak
You are so welcome, Oleg! I'm sure you'll like it.
psiphoneiphone.com
wow... sounds great eagerly waiting..
Elena Terenteva
psiphoneiphone.com
Coming soon :)
Carlos Talaga
For this improvement there will be a difference between basic users and guru? I would also like to know which countries will have the greatest impact. In my case I am interested to know about the data available for US, Australia, Spain, France and Mexico. Thanks for the post, it's a breakthrough.
Elena Terenteva
Carlos Talaga
Hi Carlos! We are not cheaters :) This update will be available for all users - on Pro, on Guru, on Business. We have all databases you’ve mentioned, and this update is rolling out not just for US database, but for all of our databases.
Hey Elena, will this new innovation avle to give me data for selected URLs and keyword or it will still how me same graph for all the URLs from the site ? I haven’t checked yet what’s new with this change. Hopefully some more usefulful data we will get.
Elena Terenteva
Ramesh Singh
Hi Ramesh! You’ll definitely get more useful data, there’s no doubt about it :) Moreover, soon you will also have the ability to check data for specific URLs (this will become available in March) - if you wan to stay tuned you can subscribe to our newsfeed. Cheers
Edward Challinor
Hi Elena, is it possible to actually suggest keywords and have them added to your database or are you saying that the new system is like a kind of baby rankbrain and will pick up on what people are searching and add them to the database automatically with AI?

For example we work in Medical Aesthetics and Dentistry and there's so many brand names and products coming out all the time. For example, Plexr is a new product that's getting like 400 hits a month to our local page alone in one tiny city in the corner of England. Is there an option to suggest new things that are breaking out. Even google trends is massively underestimating the traffic volumes for it. Says it gets 100 users a month globally but I get more patients than that in a month so it can't be right? Does it take time for even google to start to recognise volumes? It seems to be pretty accurate in search console but that's the only place i'm seeing it.

PS: love what you're doing. Since getting SEM 18 months ago my traffic has gone from 980/m to 13,000/m. That's an increase in revenue from 450kpa to 1.2Mkpa. All down to SEO that I learned reading your blogs and looking at the tools. I'd love to write a case study. I think the tool is literally magic.
Elena Terenteva
Edward Challinor
Hi Edward! First of all - thanks for choosing our tool, and I’m very pleased to hear that our blog is helping you! It would be great to have your case study on our blog, so let’s connect and discuss it in more detail.

Regarding your question - unfortunately, there is no ability to add keywords to the databases on demand. To get data about low-volume keywords, I suggest you set up the Position Tracking tool: you’ll be able to see positions and volumes for all your keywords for specific locations, even for a little town in England :)
Simon Cox
So are you saying that the SEMrush data we are currently relying on incorrect?
Elena Terenteva
Simon Cox
Hi Simon! Thank you for your question. We’ve been in this market for ten years and we have remained competitive and trusted because we are not providing crap data.

Can you rely on our data? Yes. Are we showing relevant information about your competitors? Yes. Our data has always been as accurate as our technology allowed at that time and there is always room for improvement. In the last six months, we have worked hard to improve the service architecture while analyzing and rethinking the future of our competitive analytics service in the search results. So this improvement is about moving from good to the best, not about bad turning into good enough.
Veerender
Elena Terenteva
increase in the number of brands using SEMRush will tell the story, Technology will keep improving every time that doesn't meet earlier one is bad or something wrong in it.
Elena Terenteva
Veerender
Totally agree - we are improving constantly. Thank you, Veerender!
Edward Challinor
Elena Terenteva
Simon Define 'Correct'. SEM rush estimate my traffic at about 94% accuracy. But if you look at the competitive data you can also use intuition (using your industry knowledge) to add a few percent to that. What they're saying is things change and keywords change. Some of my keywords are not in the SEM rush database so it underestimates my traffic a smidge. Hopefully this fix might include some terms that will get it colser to 100%.

This is like when google updates their core algorithm. The only difference is that unlike Google, Elena has been kind enough to actually tell us the process so we don't freak out when the results go a bit wild on 15th.

Out of interest Elena, many of my keywords are long tail and only get updated about once a month or every week or so because we are quite small and local. Will those results change immediately or will the changes be applied as the updates to my positions happen naturally over time. Some big 15,000 vol ones are every day but some of the little 420 vol maybe it's every week or so. So will it all happen at once or just as each new crawl happens?
Simon Cox
Edward Challinor
Yes, Edward, I was being cheeky - new and improved data is always better than the old data, the inference being that therefore the old data is not as good - always a headache when trying to sell an upgraded product! Anecdotal evidence and gut feel can't be measured and with SEO data the empirical evidence is hard to truly deifine causes - so many external variables are at work. Thus we need to look at trends to know how things are really performing - SEMRush certainly does that for me.
Edward Challinor
Simon Cox
do you get a discrepancy? Is it over or under? Sometimes I wonder about how SEM rush treats things when I tip over from page one to the top of page two for a really high volume keyword. Like one day 8 and the next 12 on like a 20k vol keyword (that's a lot for england) Presumably they have pretty accurate estimates of CTR at different ranks but it doesn't seem to account for what I see in analytics. Almost like it's not properly accounting for page 2 having such a diminished CTR compared witht he bottom of page 1. Does that make sense? I'm small and local so it's even harder to get it right.
Edward Challinor
Edward Challinor
Haha epic sense of humour fail. I am British though. :-/
Elena Terenteva
Edward Challinor
Hey Edward! We won’t change a schedule for keywords update - it will remain the same - depending on a keyword’s volume they will be updated daily, weekly and monthly. Similarly, you can always use Positions Tracking if you require immediate data about your keywords’ positions and volumes.
Simon Cox
Elena Terenteva
Excellent answer, thank you!
David Chadderton (Webspresso)
Sounds like this is a significant upgrade. SEMRush continues to be my no.1 go-to SEO Tool.

Are you going to make use of the extended data available from the new Search Console too?
Elena Terenteva
David Chadderton (Webspresso)
Hi David, and thank you for such a nice comment! We don’t use GSC data for keyword volume calculation. But, we are definitely keeping an eye on the GSC changes, because some SEMrush tools have GSC integration, and whenever extended data appears, there they will be shown in our tools as well.
Again a great article

Send feedback

Your feedback must contain at least 3 words (10 characters).

We will only use this email to respond to you on your feedback. Privacy Policy

Thank you for your feedback!