logo-small logo-small_halloween

New SEMrush Data: Fresh Keywords, Precise Volumes

67
Wow-Score
The Wow-Score shows how engaging a blog post is. It is calculated based on the correlation between users’ active reading time, their scrolling speed and the article’s length.
Learn more

New SEMrush Data: Fresh Keywords, Precise Volumes

This post is in English
Elena Terenteva
This post is in English
New SEMrush Data: Fresh Keywords, Precise Volumes

This year SEMrush celebrates its 10th anniversary! So let’s kick off the celebration with a big announcement and a sneak peek at a little something we like to call SEMrush 2.0.

Last year we were recognized as the Best SEO Software Suite of 2017 by the European Search Awards and by both the United States and the United Kingdom Search Awards. But perfection knows no limits.

With our upcoming innovations, we will keep raising the bar for digital marketing toolkits. And since the cornerstone of any analytical process is data, that is exactly where we are starting our renovation.

Data: What Makes It Accurate

Before we dive into detail, let's recap the main ideas and arguments concerning keyword search volumes and why they are important for digital marketers.

Volume indicates keywords’ popularity through the number of searches and is a very valuable metric for SEOs. Here it gets interesting: how do we get this data?

Google is the only official source of keyword search volumes, but over the years it grew less and less enthusiastic about sharing the information. And this is where clickstream data comes into play.

Clickstream is an anonymous record of users’ activities in web browsers and on various apps. This information is aggregated by data-providing companies and sold to all kinds of analytical platforms. Collecting this information is the easy part, processing it to get valuable results is much trickier. False volume estimation leads to faulty keyword targeting, which eventually ruins your SEO and PPC campaigns.

To ensure that you get the most relevant and precise data, SEMrush is implementing a new machine learning algorithm for search volume forecasting and validation. And we can claim to have the most accurate volume estimations on the market. Now let's see how we have achieved it.

New Data: Keeping Up With The Changes

Search is ever-changing: new popular things get the spotlight while others perish, the demand mutates, and users cultivate new habits. To stay current we have to be flexible and to be flexible we needed a change in the core of our analytical framework – the keyword database.

The first innovation affects the keywords themselves.

We are retiring old and untrendy queries and filling our database with the finest hot-off-the-press terms. Tired of shoveling out piles of worthless keywords? We are relieving you of this chore by providing only the contemporary keywords that have significance for SEO. Reinforcing our status as the best solution for digital marketers, we will deliver the most profitable traffic-driving terms and high-quality longtails.

The second change concerns volumes and the aforementioned clever machine learning method.

We will keep updating our volumes monthly, but with the new algorithm and architecture, we can raise the standards for estimation accuracy even higher.

But, like many great things, our improvement has one small drawback. A new database means new digits, which may affect your reporting. The total number of keywords can swing, and you might notice some spikes and drops in traffic estimation. We hope it will not become a major fly in the ointment because eventually, you will get the best tool with the most accurate data.

If you are concerned about the reduced number of keywords, don’t be. Our database will stay just as large, but now it will contain the most adequate keywords for website optimisation.

You will still have our Keyword Magic Tool with all of the keywords at your disposal, if you are not going after the keyword analysis for the specific domains, and need a full database for brainstorming and diversifying your semantic core. This data will also be regularly updated with the new keywords and freshest metrics.

Data 2.0: The SEMrush Way

Now let’s enter spoiler territory with a promised teaser of our new Domain Analytics. To provide the best experience we are sprucing up the interface and augmenting our reports with some remarkable features. Plus, the new architecture will immensely speed up our tool.

To optimise the SEO workflow, we streamlined our Overview Report. It will show only organic metrics with clear-cut widgets for traffic estimation, keywords, position changes, SERP features, competitors, top pages and top subdomains.

Competitors are assessed in a brand new way that is unique to the market. Managing competition research has always been our strong suit, and yet we are adding a new level of precision to the analysis. Now it is not only about the common keywords but also about rivals’ proximity in the search results. By considering the closeness of positions, SEMrush excludes contenders from the outskirts of the SERPs and surrenders to you the real opponents.

Traffic is now presented with absolute values to improve the experience of our users. Be aware that these are still estimations based on volumes and average CTRs. While the numbers are not exact, they are close to the real picture and accurate in terms of correlation. So, you should not perceive this estimation as the ultimate data source, but rather as a strong asset for comparison and analysis.

Positions and Position Changes reports have a brilliant addition. Now, all filtering that you apply will not only affect the data table but will also instantly change the graphs. This is unprecedented for such a huge database. Having a visual representation on the fly will immeasurably enhance your analytical process.

SEMrush Domain Analytics instant graphs

Our advanced filtering with instant graphs creates an incredible sandbox for nuanced trend analysis. Evaluate in a multitude of ways how the website is performing over time, assess specific subdomains and pages, and filter them out by various SEO metrics and keywords of different types.

And that’s not all, folks. More great improvements are on the way, so stay tuned.

Like this post? Follow us on Feedly and read more interesting posts:

Follow
Elena Terenteva, Product Marketing Manager at SEMrush. Elena has eight years public relations and journalism experience, working as a broadcasting journalist, PR/Content manager for IT and finance companies.
Bookworm, poker player, good swimmer.
Share this post
or

Comments

2000 symbols remain
When will finish the update? Thanks.
Did the update happen on the 15th or just start on the 15th? If it just started on the 15th how long do you anticipate before the update is done. We use your keyword rankings as one measurement for SEO and it's hard to judge if it's something Google or SEMrush is going doing without knowing when SEMrush update is complete. TIA
Hi, Its a week ago but in all sites of my industry I still see for all of them keywords loss of 50-60%. When will finish the update? Thanks.
Jose
I second that, most domains tracked show a loss of keywords.
Great post, thanks. Question: when will the process of updating the keyword database end (or stabilize to some extent)? I'm using the "number of keywords my site is ranking for" report as an indicator of how things are going - if numbers are growing fast, we're doing good. If they're falling slow, it's meh... if they're falling, something's wrong and we need to act. So far, within the last 20 or so months, we fell once for a few days in August last year (and indeed, there were two problems with the site - we fixed them and started growing again) and the 2nd time... yesterday. I'm hoping this is due to the database overhaul, but I'd love to know:
* are you dropping some keywords from the database? If yes, how significant the drop is (percentage of the whole DB would be great).
* how long will it take before the DB stabilizes and doesn't change much anymore.

Side note: I've always missed one single thing from the report - a total number of keywords - I was never sure if our "growth" was due to us REALLY doing better or was it because you introduced some new keywords that we rank for.
At what time today does this update get pushed? Thanks.
Martin Woods
The update instant update to graphs is a brilliant feature, I look forward to trying it out!
Elena Terenteva
Martin Woods
Hi Martin! Please give us your feedback later, looking forward for it.
David Iwanow
Wonderful news :)
Elena Terenteva
David Iwanow
Hi David! Indeed, the update is starting today - let us know what do you think.
Darvin Reyes
This is a surprising information, thank you ms, Elena!
Elena Terenteva
Darvin Reyes
You are very welcome, Darvin! We'l continue to surprise you this year :)
Thank you for the informative article, Elena! Over what period should we expect these changes to occur - and when (roughly) can we expect the full changes to be complete? Thanks!
Elena Terenteva
Alex Sturwold
Hi Alex! The update of our largest databases is starting on January 15, today we’ll provide additional info about it. And this will take up to one month to roll out. Smaller databases are already being rolled out.
Oleg mak
Cool news, thanks for your work
Elena Terenteva
Oleg mak
You are so welcome, Oleg! I'm sure you'll like it.
psiphoneiphone.com
wow... sounds great eagerly waiting..
Elena Terenteva
psiphoneiphone.com
Coming soon :)
Carlos Talaga
For this improvement there will be a difference between basic users and guru? I would also like to know which countries will have the greatest impact. In my case I am interested to know about the data available for US, Australia, Spain, France and Mexico. Thanks for the post, it's a breakthrough.
Elena Terenteva
Carlos Talaga
Hi Carlos! We are not cheaters :) This update will be available for all users - on Pro, on Guru, on Business. We have all databases you’ve mentioned, and this update is rolling out not just for US database, but for all of our databases.
Hey Elena, will this new innovation avle to give me data for selected URLs and keyword or it will still how me same graph for all the URLs from the site ? I haven’t checked yet what’s new with this change. Hopefully some more usefulful data we will get.
Elena Terenteva
Ramesh Singh
Hi Ramesh! You’ll definitely get more useful data, there’s no doubt about it :) Moreover, soon you will also have the ability to check data for specific URLs (this will become available in March) - if you wan to stay tuned you can subscribe to our newsfeed. Cheers
Edward Challinor
Hi Elena, is it possible to actually suggest keywords and have them added to your database or are you saying that the new system is like a kind of baby rankbrain and will pick up on what people are searching and add them to the database automatically with AI?

For example we work in Medical Aesthetics and Dentistry and there's so many brand names and products coming out all the time. For example, Plexr is a new product that's getting like 400 hits a month to our local page alone in one tiny city in the corner of England. Is there an option to suggest new things that are breaking out. Even google trends is massively underestimating the traffic volumes for it. Says it gets 100 users a month globally but I get more patients than that in a month so it can't be right? Does it take time for even google to start to recognise volumes? It seems to be pretty accurate in search console but that's the only place i'm seeing it.

PS: love what you're doing. Since getting SEM 18 months ago my traffic has gone from 980/m to 13,000/m. That's an increase in revenue from 450kpa to 1.2Mkpa. All down to SEO that I learned reading your blogs and looking at the tools. I'd love to write a case study. I think the tool is literally magic.
Elena Terenteva
Edward Challinor
Hi Edward! First of all - thanks for choosing our tool, and I’m very pleased to hear that our blog is helping you! It would be great to have your case study on our blog, so let’s connect and discuss it in more detail.

Regarding your question - unfortunately, there is no ability to add keywords to the databases on demand. To get data about low-volume keywords, I suggest you set up the Position Tracking tool: you’ll be able to see positions and volumes for all your keywords for specific locations, even for a little town in England :)
Simon  Cox
So are you saying that the SEMrush data we are currently relying on incorrect?
Elena Terenteva
Simon Cox
Hi Simon! Thank you for your question. We’ve been in this market for ten years and we have remained competitive and trusted because we are not providing crap data.

Can you rely on our data? Yes. Are we showing relevant information about your competitors? Yes. Our data has always been as accurate as our technology allowed at that time and there is always room for improvement. In the last six months, we have worked hard to improve the service architecture while analyzing and rethinking the future of our competitive analytics service in the search results. So this improvement is about moving from good to the best, not about bad turning into good enough.
Veerender
Elena Terenteva
increase in the number of brands using SEMRush will tell the story, Technology will keep improving every time that doesn't meet earlier one is bad or something wrong in it.
Elena Terenteva
Veerender
Totally agree - we are improving constantly. Thank you, Veerender!
Edward Challinor
Elena Terenteva
Simon Define 'Correct'. SEM rush estimate my traffic at about 94% accuracy. But if you look at the competitive data you can also use intuition (using your industry knowledge) to add a few percent to that. What they're saying is things change and keywords change. Some of my keywords are not in the SEM rush database so it underestimates my traffic a smidge. Hopefully this fix might include some terms that will get it colser to 100%.

This is like when google updates their core algorithm. The only difference is that unlike Google, Elena has been kind enough to actually tell us the process so we don't freak out when the results go a bit wild on 15th.

Out of interest Elena, many of my keywords are long tail and only get updated about once a month or every week or so because we are quite small and local. Will those results change immediately or will the changes be applied as the updates to my positions happen naturally over time. Some big 15,000 vol ones are every day but some of the little 420 vol maybe it's every week or so. So will it all happen at once or just as each new crawl happens?
Simon  Cox
Edward Challinor
Yes, Edward, I was being cheeky - new and improved data is always better than the old data, the inference being that therefore the old data is not as good - always a headache when trying to sell an upgraded product! Anecdotal evidence and gut feel can't be measured and with SEO data the empirical evidence is hard to truly deifine causes - so many external variables are at work. Thus we need to look at trends to know how things are really performing - SEMRush certainly does that for me.
Edward Challinor
Simon Cox
do you get a discrepancy? Is it over or under? Sometimes I wonder about how SEM rush treats things when I tip over from page one to the top of page two for a really high volume keyword. Like one day 8 and the next 12 on like a 20k vol keyword (that's a lot for england) Presumably they have pretty accurate estimates of CTR at different ranks but it doesn't seem to account for what I see in analytics. Almost like it's not properly accounting for page 2 having such a diminished CTR compared witht he bottom of page 1. Does that make sense? I'm small and local so it's even harder to get it right.
Edward Challinor
Edward Challinor
Haha epic sense of humour fail. I am British though. :-/
Elena Terenteva
Edward Challinor
Hey Edward! We won’t change a schedule for keywords update - it will remain the same - depending on a keyword’s volume they will be updated daily, weekly and monthly. Similarly, you can always use Positions Tracking if you require immediate data about your keywords’ positions and volumes.
Simon  Cox
Elena Terenteva
Excellent answer, thank you!
David Chadderton (Webspresso)
Sounds like this is a significant upgrade. SEMRush continues to be my no.1 go-to SEO Tool.

Are you going to make use of the extended data available from the new Search Console too?
Elena Terenteva
David Chadderton (Webspresso)
Hi David, and thank you for such a nice comment! We don’t use GSC data for keyword volume calculation. But, we are definitely keeping an eye on the GSC changes, because some SEMrush tools have GSC integration, and whenever extended data appears, there they will be shown in our tools as well.
Again a great article