English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語

All you need to know about Google Penalty, Toxic Score and SEMrush Backlink Audit




Tristam: We're live. Hey everyone. Good morning. Good evening. Good afternoon, wherever you are in the world. Today we've got Sandeep, Ilya, Lina all joining us for this mega bonanza of a webinar.

Lina: Hi.

Ilya: Hi there.

Sandeep: Hi.

Tristam: Thanks for everyone tuning in. I see. We've got India, we've got Seattle, Mongolia, Baltimore. It's great to see the community coming together as always. And it's such a worldwide community. 

Over to you Ilya. I don't know if you want to just quickly intro the piece that you're going to be doing and then as you're doing now, share your screen.

SEMrush Backlink Tools: Overview

Ilya: Okay. Hey guys, my name is Ilya. I work in SEMrush as a product owner of the Backlink Analytics tool. What do we do? We find all those links, and we build a couple of tools that help you to analyze that huge amount of data. 

What we’re going to talk about today is links. What are the tools we have? What is our new authority score? Which might be very important in terms of link building.

Just a few words about the database. There is so much data we have. Just to give you an overview, we crawl around 17 billion URLs per day. We have a lot of data and we are one of the top backlinks providers on the market. We have only one index. Each competitor of ours has different terms of how they interpret their data.

We keep links for six months, but if they are lost or we haven't seen them for more than six months, we're going to just get rid of them because we think that they are not really important. If we haven't crawled them, probably there is some rubbish. 

The big question is who has the biggest database? I know there's a lot of people who have their own blogs right now listening to this webinar. Some tools, they store all the data from 2014. As I have mentioned, we store data for not more than six months if it's become irrelevant. 

Even the meaning of a link is different. Some tools say there’s only one link from one URL, and any others we won’t show. 

There's some big websites that we can crawl, for example, every day. Some other websites we don't think they are that important so we crawl them once in two weeks, once in a month. That's why for some short time periods there might be some differences between the database sizes. 

SEMrush Key Backlink Metrics

Let's talk about SEMrush advantages. First of all, we have different backlink tools. There's not only Backlink Analytics; there's Batch Analysis, Backlink Gap, Link Building, and Backlink Audit. Not any other tool on the market can say they have so many tools to use. We have our own unique metrics. We're going to talk about that on the next slides.

SEMrush is not only a link tool. We have keyword data, we have clickstream data, and we all combined that to make our metrics more sophisticated. We have some easy reporting, just one click you can have a nice PDF report if you're just working for a client or want a nice screenshot. 

And the speed of the interface. Even if you were going to go for amazon.com, just go for it. You're going to wait just a couple of seconds and you can see the link itself. 

All right, so what metrics do we have? Why are they so important? First I will talk about the basics. What is new? What is lost? What is authority score? What is toxic score? What are monthly visits and keywords? 

The basics, what is a link? Not everything can be counted as a link. There might be the UTM codes, there might be the URL parameters. Multiple links might be just splashed into one. 

Also we respect the robots.txt, the noindex files. Some people usually come up with issues like, "Why this thing hasn't been indexed?" It's been here for a while so we can't index that.

We always have questions like, "My Google Search Console, I can see a lot of links and you have only 12." The difference is that every tool has its own method of crawling. We crawl some parts of the internet more and other tools might crawl some other parts of the internet in different ways. 

It's why I will just recommend you to double-check it in your Google Search Console. Or in our tool that is called Backlink Audit, you can connect the Google Search Console and have all the databases together. 

What is a new link? A new link is a link that we saw not more than one month; it is one month old. A lost link; this is something that we couldn't crawl for the second time because of some technical issues, for example. Or it might have just disappeared.

Let's go to the authority score. What is that? I think we're all familiar with the Page Rank. There is a link juice and all the referring domains pass that juice to the target domains. That's a classical page run. 

We made this algorithm, but also we improved it a little bit. I already mentioned that we know how different websites are ranking in organic search and Google. That's why we know the list of sites that we believe they're trusted. If they rank fine in Google, they have some certain amount of organic traffic, we would say they are trusted websites or links from those sites would be much more powerful. 

These authority scores might go down, even if nothing is changing. What does that mean? What we basically do, we have all the websites that we line up from the first one to 12 trillion and after that we normalize it into a scale of zero to 100. Something closer to 100 is better. 

If nothing is changing, like you have the same amount of links for years, you might go down. Why? Because the domains that are close to you, their link profile becomes stronger and stronger. That's why other websites go up and that's why you go down. 

Then what is a good authority score? What is good for link building? I would say the one that is comparable to yours is basically good for link building.

Why? Of course, you probably want to have a link from websites that have authority score 80+ or 70+. But if you got 10, I wouldn't say so if you have a lot of chances to get that link. If you have vice versa website, which is 80+ authority score, you probably don't want those links from their sites that have 5 or 10, especially if they are paid ones. 

Probably one of the most just famous questions that me and Lina receive, “there is authority score and it's 20 or 30 or whatever, but it's toxic”. How is that? They do not correlate with each other; they are different scores. 

One would say, "You guys have a toxic profile, you need to do something then," and the other one, "How you are likely to have this link," so they don't have to correlate. You have to look at both scores to have the full picture. 

There are around 1,000 domains which have authority scores of 80+. If you have a domain 80+, congratulations, you might be a very rich person. There are around 10 million domains with authority score which is less than 20. 

The less the authority score is, the more domains with the same score there is on the web. If you had authority score from zero to 10 the average number of the referring domains you have is 40. If you had an authority score, for example, 30 or 40, there are around 800 referring domains you have. Of course, the authority score and the number of domains they do correlate. 

Is it everything? I wish I could tell you an authority score of 50 is fine, go for it (the link). But one number cannot tell you the full picture. 

I believe we have just a couple of minutes for the questions. Is there any?

Diagnosing Toxic Link Scores and Disavowing Image Links

Tristam: Yeah, there's quite a few. If I start from the top and work our way down. OD AC asked, “if SEMrush is giving a high toxic score, but the domain isn't really toxic, what should we do?”

Ilya: If there is a high toxic score, that means that a lot of referring links, referring domains are unnatural or there is some fishy anchor text or any other spam signals. You probably had some attack or something like that. I would just recommend to set up a backlink audit and just have a look what toxic domains you have and disavow them.

Lina: Backlink Audit it's my tool, and let me add something. In case you truly believe that the domain that was marked  “toxic” is not toxic, you could easily just whitelist it in the tool and this way you will tell us that it's not bad domain. Maybe our algorithm next time will align better. It's also possible just to whitelist it but only in case you definitely believe it's not toxic

Tristam: Perfect. 

Ilya: All right, thanks Lina.

Tristam: Fantastic. I'll see if I can get through a couple more questions and then we'll jump into your presentation Lina. We've got Care for Specialty Pharmacy. "Sometimes my backlink audit shows sites using images from my site. Is it important to disavow them or can they be ignored?"

Lina: It definitely depends on the kind of images and we should take into account whether it was any manipulation under the use in that image. We definitely should deep dive into your project to see your case.

Ilya: I think disavow questions are really sensitive. It's almost impossible to tell you should disavow something or not without actually looking into those things. 

Tristam: Just for the audience to understand. There's no kind of “one rule applies to everything” really when it comes to disavow and backlinks. It's really about individual cases because there's, I guess so many little pieces of mechanics going on that you can't say “this is definitely it and you should always do that”. Really you should always dive into your data and explore a little bit further to find out, "Is this a good or bad link," or before you even go down the road of disavowing.

I guess we're at time now. Lina, if you're ready, I will hand over the gauntlet to you and when you're ready share your screen.

Google Link Penalties: Overview

Lina: Okay. Great. First of all, thanks to everybody for coming to our webinar. I'm happy to continue our topic connected to the backlinks. My name is Lina. I'm also from SEMrush and Ilya thank you for the short introduction. I got two products: Backlink Audit and Link Building Tool. My team actually developed these two products.

Today I propose to talk about a really intriguing topic relating to backlinks: unnatural link penalties. In my presentation, I would like to share with you some findings. How to avoid the penalties is the most important thing, I would say. 

We all know that backlinks are quite important to our ranking. It's one of the main Google ranking factors. That means that we can't just simply ignore them. We can analyze them, treat them well. 

I really like the comparison of links with the votes. If you get more links, you're getting emails, votes from different people saying, "Hi, this page is good. This content is really good, let's check it out." From the search engines’ perspective, it's votes that could determine that people actually want to view or read that content.

Actually I should mention that not all backlinks are equally good for your site. Some of them could be manipulative or unnatural or some of them could hurt your website. For example, it'll negatively affect your traffic, ranking, or even revenue as a result. 

How to understand if a backlink is bad? We know that simple rules from Google are present in the Google Webmaster Guidelines, whether a link is good or bad. I really recommend you go to this document first; it gives you really good clues where to start with your link assessment. 

We also know that Google with its algorithms like Google Penguin tries to fight spam links. Google still broadly uses manual verification of your website, which can lead to Google manual action or penalties for your website. 

A sign of the manual penalty is that someone from the Google Webspam team has reviewed the website manually link by link. I don't know exactly how they do that and actually check whether the website complied with the Google Webmaster Guidelines. 

We see the quotes, I know it was from Google. It's a quote from the Webmaster blog. The article was published in 2019, and they told us that four million messages were sent connected to the manual action in 2018. It still exists. 

You also asked how you could detect that you got penalized by Google. It's quite easy. You will always receive a notification in your Google Search Console or you will also receive email with some information about your penalty. You will be notified about what kind of penalty you got. 

Some penalty could be applied to the part of your website, to some section, property or particular page. If you are lucky, in that notification, you will receive some example links like the precise link that Google counts as manipulative or toxic. Sometimes it’s just a notification. What else you will notice after you get a penalty is a drop in your metrics.

One indirect way to detect that your project or website is at risk is to conduct the backlink audit. In SEMrush in Backlink Audit we have a special algorithm that will calculate the possible risks to be penalized by Google. If you see the high toxic score, it means that you have a lot of unnatural backlinks in accordance to our algorithm and they should be at least revised by you. 

There are two types of manual penalties, for inbound links and separately for outbound links. I would say the case when you receive the penalty for outbound links that are placed on your website is easier to handle because it's your own website, you can just get rid of that link quite fast. 

But it's not the case for inbound links because these links are not under your control. Sometimes you even don't know that you have such a link. In most cases, in order to deal with that kind of penalty, you need to use special tools for conducting deep analysis and treating those links.

In my presentation, I will focus mostly on cases that connect you to inbound links because it's really hard to deal with. As I already mentioned, we have a special tool, Backlink Audit, that conducts in-depth analysis of your profile. It's quite an old tool. Believe me every year, every month, we're trying to improve it.

Common Reasons for Link Penalties

During the last couple of years, we collected and analyzed a lot of different penalty cases for the main purpose to improve our algorithm, and we do it constantly. I think our findings could be really interesting to you, to the broader audience. 

Let me share some results. First of all, you'll see that the first two places are penalties for sponsored or paid links and guest posts. Then we see in the third place, fourth place mostly manipulative techniques like PBNs, user-generated spam and forums, and comments for blogs or low-quality directories. 

I should stress that we didn't see any penalty for links that were placed on the high-quality directories or all the business directories. Usually, it's connected to the low-quality resources. Link widgets, links in footers still take place and you still could get a penalty for such cases. 

A lot of guest posts were penalized. Actually also a really, really interesting case, according to the Google rules, when you have a sponsored link, you paid in some way for the link, you should nofollow an attribute. At the same time, you need to apply the disclosure, sponsored disclosure. 

It was actually a very interesting question whether you could get a penalty for one, two, three, five links. We didn't see any cases. Usually, it was widespread manipulation: a big portion of your web backlink profile was of manipulative backlinks. 

Lifespan doesn't affect the toxicity of links. It's also why they believed that you could be penalized only for the fresh, new backlinks. You should take into account all your historical profile. Reputation doesn't matter, at least to Google. Could a high authoritative website be toxic? Yes. It could be. 

Exact match anchors need to be treated really carefully. Most of the cases we analyze contain exact match or “money” anchors. Definitely Google pay attention to them. In most cases, you are on your own. 

Recovering From a Google Penalty

It's not so easy to recover from a penalty. It usually takes a couple of reconsideration requests until your projects reconsideration requests will be approved. Some projects could be penalized for five, six, or even more months. It could be really harmful for your business. 

What to do in case you got a penalty? What can we advise you? The main stage is trying to collect as much information as possible. Focus on the real scope of your penalty, because not always, you got penalized site-wide; in some cases, you get a penalty for some specific page, section, property.

After that conduct in-depth analysis, trying to classify your links. Paid links, you need to deal with them separately, mostly trying to change their relative view to just sponsored link or nofollow and spammy backlinks that should be removed or disavowed accordingly.

It's really a process because you need to manually check every backlink and decide whether it's truly toxic, truly spammy or not. We are trying to help you with that by providing you with a toxic score. It's our special metric for every backlink. It shows you how probably these links are toxic or spammy. 

All the other backlinks that you can't manage to remove you should disavow. Disavow it's like a special tool that's not connected to the Google search console. It's a separate tool. You should create the TXT file based on the toxic domains or special URL you would like Google to ignore and upload to Google. 

After that, collecting all of the detailed information about your toxic backlinks, why you decide that these backlinks were toxic. All information about your outreach process and disavow file. All this information you will send a reconsideration request to Google and wait for one, two, three, sometimes more weeks until someone from Google will review one more time your project, and you still have a chance that you will cross to the second round of the reconsideration request. 

It's better to prevent, and even John Mu told us that he recommended cleaning up the toxic or spammer backlinks before someone from the Spammer Team does it for you. How to do this? Just like with the backlink audit tool. You can create a project and it will automatically audit your project on a regular basis and just to pay attention to new suspicious cases, and they could get rid of really pure spam and keep an eye on your profile. 

The most important thing is to invest in good, really quality backlinks. I think these facts will be covered by Sandeep in his presentation. It's really important. Thank you, guys. That's it from my side. 

Tristam: Fantastic. Thank you very much Lina. If you could stop sharing your screen, that'd be awesome. Keeping in mind the time, Sandeep, I think we'll jump into your section. Then with any time left, we can recap and cover any questions that we haven't been able to. 

Sandeep: Yeah. This is going to be a Q&A section, right? You can probably start with the first question. 

Best Strategies for High-Quality Backlinks

Tristam: My first question for you Sandeep is you've got a great piece on backlinks where you explain step by step how to get higher quality ones. Your article is very popular. Can you let me know, and the audience, what your two favorite link building techniques are out of the 25 that you've got here? 

Sandeep: Sure. I wish I could talk about all 25, but my role in this whole presentation is going to be that of a cameo. Ilya and Lina were the superstars. Let me just quickly jump in and talk about my two favorite link building strategies. 

Natural links are the best kind of links. These are the kinds of links that Google wants you to build, and this is what we should be aiming to do. One of the best ways to do it, at least from my own personal experience is to create evergreen content. 

Evergreen posts are posts that stay relevant for a long time after they have been published in contrast to seasonal topics, like say, if you're creating content on COVID-19. You'll see a lot of marketing articles which feature COVID-19 prominently. These keywords may have a high search volume, but they'll probably not have the same search volume a year from now when hopefully the pandemic has been resolved.

You should be aiming at creating evergreen content. Content that stays relevant for a long, long time after you publish them. Now it's easier said than done, right? I mean, we can all say, "You need to publish evergreen content”. But there are certain ways you can make sure that your content stays evergreen.

The first step in creating evergreen content is to find keywords, find evergreen keywords. This is where you can use the SEMrush Keyword Magic Tool. You can find a lot of keywords, keyword variations that you can use in your content, right? 

That's one tool that I would recommend and make sure that you stay away from seasonal keywords or pop culture trends because these keywords will probably have a search volume in the short run, but in the long run, you can't expect them to sustain the same level of traffic that you'll probably get now.

The second step is to create what we call in the marketing world, “10X content”. What I mean by 10X content is your content needs to be 10 times better than the top ranking posts for your target keywords. If you create content that is 10X better, chances of you getting outranked are much lower. That's what we say when you need to create content that is 10 times better. 

How do you do that? You need to bring your own unique perspective, you need to talk about your own case studies or experiments and whatever unique insights that you can add. Something that's currently lacking in the top three search results. You can bring that out in your article and that's how you create a 10X content. Now that's the first way to build links, right? You create evergreen content. 

In my opinion, the second-best link building technique is to steal your competitor's backlinks. Now when we talk about natural links, again it's easier said than done. Your evergreen content will probably fetch links, but it'll probably take a long time before you have reached the same level as your competitors. 

This is where you need to study your competition. This is where you need to understand the kind of backlinks that they have developed over time. This is where you can use the SEMrush backlink analytics tool. You can just type the URLs that are currently on top. You can see the kind of backlinks that they have generated, and you can just make a list of all the backlinks that you would like to target. 

Ilya already spoke about the authority score. Make sure that the backlinks that you are targeting have a decent authority score and make sure you use the right filters in SEMrush when you are targeting these backlinks. 

When you have a spreadsheet of all the link prospects ready, you can use a tool like Ninja Outreach, or Find That Lead to mine the email addresses that you would need to perform your outreach campaign and send personalized emails to your link prospects...and highlight pieces in your content that are better than your competitors. Give them an incentive as to why they should link back to your content. 

Those are my two favorite link building strategies, and I wish I could deep dive more into these tactics, but we are running out of time. 

Creating Evergreen Content That People Link To

Tristam: Fantastic. Thank you very much. So apart from some of the areas that you've covered in what you were just discussing there, what's your kind of usual workflow for creating such pieces? 

Sandeep: Like I said, whenever I create content, I try to stay away from pop culture trends or trends that are probably popular in the short run. 

One particular case study that I published recently was how I keep updating my articles. A lot of people, what they do is once they have published their article, they just share it on social media. They share it on different marketing channels. They share it with their email list, but it's just published and they forget about it. 

What I would advise to every blogger out there is to keep updating your articles. Because you are going to be outranked sooner or later because the competition is so high, especially in the marketing world where there are so many bloggers competing for the same keywords. You need to keep updating your content with fresh information. 

You need to add unique insights. You need to change the screenshots. You need to change the images. You need to add videos. Just make it as unique as possible. By doing this, what I've achieved is I've achieved exponential traffic growth. 

Whenever I find a declining trend in the traffic of some of my articles, some of my top ranking posts, I revisit them, update them with new information. As you can see on the screen, this is what I've achieved. In the last 30 days, I was able to boost the traffic of one of my top ranking articles by 146%. 

The other one, which is not ranking at all, which was probably buried on page two or page three, I was able to increase the traffic by 693% for my target keyword terms. That's what I do. I just keep relaunching my content. I keep updating it so that the insights that you can see in my articles are fresh and something that's really valuable.

Tristam: What would you usually do first in terms of, would you analyze and use a product on your own to write a high-quality piece? Or is there anything else that you would get involved with?

Sandeep: Yeah. If I'm publishing affiliate articles on my blog, if I'm talking about a product, if I'm doing a review of a product, if I'm comparing it with some of the other leading products, I try to understand the product inside out. I try to understand the strengths and weaknesses. I need to be the user first, right? 

I get a deep understanding of the different features and functionalities, and that's when I write a review and I would probably recommend this to everyone who's doing a review or a how-to guide especially when it comes to affiliate marketing.

How to Promote Articles in a Non-Spammy Way

Tristam: Did you take any specific actions to promote this article or any of your articles in that manner? 

Sandeep: Aside from publishing them, aside from updating them on social media, promoting them on social media, and then promoting it to my email list. What I typically do is I participate in these different Subreddits, right? I also participate in these marketing forums, like GrowthHackers, and I also take part in Quora discussion. These are indirect ways you can promote your content. 

I'm not saying you should just jump into a discussion and just place your link there. That's not how it works. You need to be very cautious when you're participating in a Reddit Forum or a Subreddit as it's called, especially the SEO Subreddit because they can smell self-promotion from a mile. 

There are so many link building questions. There are so many questions that pertain to SEO, and you can't provide an answer in the short Reddit comment, right? You can always link to your content. You can say, "Here's my answer. If you need more details, please visit my article where I talk about this in more detail." You need to be very subtle when it comes to promoting your content. 

Beginner Content Creation Tips for Affiliate Marketing

Tristam: What would you recommend a newbie in content creation to do to become successful in content creation and affiliate marketing in particular? 

Sandeep: Yeah. What I would recommend to affiliate marketing beginners is first build an audience. You can't start a blog or a podcast or a YouTube channel and start promoting affiliate products right away. You need to gain the trust of your audience, and no one is going to buy from you if you have not established some amount of expertise.

I'm not telling you that you need to build an audience of half a million visitors. That's not going to be achievable. It's not an achievable target in the short run. Even if you have an engaged audience of say 5,000 people who regularly visit your blog, if you're getting consistent traffic of 5,000, 6,000 users every month, that's when you can start affiliate marketing simultaneously.

When you're creating content, when you're creating different kinds of content to establish your authority, you can do affiliate marketing simultaneously. When you start with affiliate marketing, just focus on one to two products first. I've already spoken about how important it is to understand your product thoroughly before you start creating content on it. 

Understand the product, become an expert, know more about the product than the product manager himself before you create content, because you're going to get a lot of questions on your blog. Whenever you publish articles, whenever you publish reviews, whenever you publish comparison articles, you're going to get a lot of questions, and you need to answer them as best as you can. You can do that only when you have a certain level of expertise in the product that you're writing on. 

Be very focused when it comes to affiliate marketing, just select one to two products first, and it's very tempting to join as many affiliate programs as possible. I've already said about marketing tools how almost 90% of them have an affiliate program. It's so tempting to join them and sign up for free trials and write reviews.

But when you do this, you probably won't move the needle, you won't see any growth in your affiliate income, and then you'll be discouraged. Then you'll just give up on affiliate marketing. Don't do that. Just be really focused on one to two products and create different content pieces, create how-to guides, create reviews, create comparison articles, create video tutorials, even do a podcast episode on your favorite product.

Even when you're doing affiliate marketing, it shouldn't seem like you're doing affiliate marketing. You have to be very subtle when it comes to doing all these comparison articles and then don't be shy to highlight a few weaknesses that your favorite product has. That will just make your audience trust you more when you do this. 

Tristam: Sadly we are over the hour. Guys, I know there was a hell of a lot of questions that came in that we just haven't had a chance to answer. Sandeep, Lina, Ilya, how can anyone get in contact with you with maybe a technical question or questions that we just weren't able to answer. Sandeep if you've got some social you want to throw out there for everyone?

Sandeep: Yeah. You can always get in touch with me on Twitter. You can send an email to sandeep@99signals.com. You can visit my site 99signals.com. And if you have any questions, you can just use contact form and just share your question and I'll try my best to give the best answer possible. 

Tristam: And Lina, how can people get in contact with you?

Lina: Actually the easiest way honestly is to write a message. Drop me a message with a feedback form on the backlink audit or via LinkedIn.

Tristam: Perfect. And Ilya, how about you?

Ilya: I just sent my email to the chat, so you can just drop me a line or you can add feedback within our UI. Honestly, we read it all the time.

Tristam: If you're watching this back, thanks for watching it back. That's awesome. Again, I apologize we weren't able to answer everyone's questions. I don't know where the time goes with these things, but for now, I just want to wish everyone a good morning, good afternoon, or good evening, wherever you are. Until next time, bye for now.

Ilya: Thanks Tristam for being an awesome showman.

Sandeep: Thanks everyone.

Tristam: Thank you.

All levels