Sometimes you make something better not by adding to it, but by taking away.
When I got into the SEO game about 10 years ago, I remember being taught that it is important to pump as many pages into Google’s index as possible. That really, really doesn’t work anymore nowadays.
Panda and other Google algorithms have taught us that quality - not quantity - is the name of the game. An effective approach to keeping quality high is “SEO pruning”: cutting off or editing underperforming pages to make a site stronger. You would rather have little but great content.
That’s what SEO pruning is all about.
The Idea Behind SEO Pruning
The concept of pruning is taken from Bonsai trees. They tend to grow to the top, which means the outer branches grow out of proportion, while the inner branches get crummy and die off. Pruning can get a Bonsai tree into the desired shape by making sure growth is distributed more equally.
The same applies to websites and SEO performance.
Most sites follow a power curve: a small number of pages gets most of the traffic. That in itself is not bad, but it results in a long-tail of underperforming pages that don’t get enough “light”. They are the crummy branches. What you want instead is a high ratio of indexed pages vs. pages with top 10 rankings. Sites with a low amount of underperforming pages seem to get a sort of extra boost.
We don’t know what the exact mechanics behind pruning are, but it seems to be a mix of optimizing crawl budget, quality score, and user experience. The original idea of a quality score comes from paid search, but the concept also exists in relation to a site’s link graph or brand combination searches. For content, Google seems to measure the quality of all pages in its index and accrue it to a "domain quality score”.
Now, the question is how can you prune your site and what tools should you use?
Let’s take a look!
How to Do SEO pruning with SEMrush
The goal of this step-by-step guide is to create a spreadsheet that tells you which URLs are underperforming, so you can prune them. I am using one of my personal projects in Germany to lead through the tutorial.
What the final result should look like. Hat tip to conditional formatting.
Before we get started, make sure you connected Search Console and Google Analytics (or your web analytics tool of choice) to SEMrush and have a project set up for the domain you want to prune.
Step 1: Crawl Your Site with “Site Audit”
First, we need to crawl our site, so go to “Site Audit” under “Projects” and get one started.
The “Site Audit” feature in SEMrush: Dashboard -> Projects -> Site Audit.
Once the crawl ran through, you get a report that lets you export the crawled pages (on “crawled pages”, click on the export button in the top right).
Crawled pages report in the Site Audit feature.
Export and save the data.
Step 2: Extract User Behavior and Backlink Data
Now we have a basic index of pages, but we still need the data to assess which ones to prune. We can use many different criteria to assess the performance of a URL:
- Organic entrances
- % bounced entrances / bounce rate
- Total pageviews/sessions
- Number + quality of backlinks
- Crawl frequency
- # of ranking keywords
- CTR
- Backlinks
- Social shares
In my example, I used # of ranking keywords, backlinks (from SEMrush and Search Console), traffic, user signals. “Organic Traffic Insights” has all of that (can be found in the projects section).
The “Organic Traffic Insights” report in SEMrush.
Don’t forget to define the time range before you export. By default, it is set to 7 days, but that is not very representative. Instead, choose the last 3-6 months.
Pick the right time range for the data export.
What you get out should look something like this:
Export from “Organic Traffic Insights” in SEMrush
What is nice is that the export gives you get average user behavior metrics across your site to benchmark. Plus, below the URL overview, you get all ranking keywords for each URL*.
*(With the latter you can create a pivot table and check whether several URLs rank for the same keyword to assess potential keyword cannibalization. But that is a topic for another blog article.)
We also need to export backlinks, so head over to “Backlink Analytics” -> “Backlinks” and export ‘em all. Just make sure you exclude lost backlinks before the export, otherwise, your list will be noisy.
“Backlinks” in SEMrush
Additionally, you can add backlinks from any tool you like, e.g., Search Console. The goal is to get a number of backlinks per URL, or a backlink quality metric.
In Search Console, just click on “Links” -> “Top linked pages - externally” and export that list.
External links report in Google Search Console: Status -> Links -> Top linked pages - externally.
Once you downloaded all the data, import the CSVs into the spreadsheet from the crawl in step 1. Then, use VLOOKUP to group all data per URL. Create a pivot table for the backlink data, so you can group the sum (or average strength if you use a proprietary metric) of backlinks per URL.
You should now have a list of URLs with information about rankings, traffic, and user behavior performance.
I indicated which data comes from SEMrush (orange) and which from Search Console (grey) in my spreadsheet:
The final pruning table sorted after links from search console.
Step 3: Identify Underperforming Pages
From here on it’s all Excel magic.
Apply conditional formatting to each column and sort the table after traffic to see why certain URLs are not performing as well. Sometimes, a page has a high bounce rate, other times not enough links, or it was never optimized for a keyword.
What you want to do is take the whole list and group URLs into three buckets.
Performers (do nothing)
The first bucket is for pages that are performing well. They rank (well) for keywords, bring in traffic, have good user signals and backlinks. So, you obviously don’t want to prune those. Pages that cannot be removed also fall into this bucket, like homepage, Terms of Service, feature landing pages, etc.
Slight Underperformers (edit)
Slight underperformers bring in some traffic but don’t rank for any keyword in the top 10. They could get on the first page on Google with some optimization. They might also have brought in a lot of traffic once but dropped in rankings over time.
Definite Underperformers (merge or redirect)
The last group is reserved for pages that never got any traffic or backlinks. They are sometimes duplicates or overlap with other content, e.g., two articles targeting the same topic/keyword. Often, they are just pages that are part of the infrastructure and have never been optimized for a (good) keyword, e.g., archive, author, tag pages.
Pages that often fall into this bucket are:
- Author/tag/archive pages
- Blog articles
- Hub pages (from topic clusters)
- Empty inventory (products that aren’t available anymore)
It is best to redirect those pages either to another one that is similar (maybe targets the same keyword) or to the next higher page in the hierarchy (could be the blog overview page or a category page). You can also merge the content with another page, depending on how well that is feasible.
Step 4: Determine What to Do with Underperformers
Pruning doesn’t mean deleting URLs. You have a couple of options for what to do with the 2nd and 3rd bucket:
- Merge the content of two or more URLs into one. Make sure to 301-redirect the old URLs to the new one.
- Redirect a URL to another or the page next higher in the hierarchy.
- Edit/rework/rewrite/update the content.
Start by setting all URLs in the underperformer bucket to meta=noindex. You might not be able to change all pages, for example when it is an author page that you need people to be able to click on. But you can prevent Google from indexing it, which already helps.
Then, go through the list of underperformers (bucket 3) and decide what to do with each of the pages you can change (merge, redirect, or edit).
Only then would I start working on the second bucket, the slight underperformers. In some cases, it might not even be necessary to prune slight underperformers.
Step 5: Release Changes in Stages
At this point, you just have to make the changes and roll them out.
I recommend releasing in stages: start with underperformers and wait for a bit (3-6 weeks). See what happens and make a data-informed decision. If the change is positive for your site, you can try to continue with the slight underperformers.
You don’t want to prune too much at once. Try to find the optimum, instead.
What results can you achieve with pruning?
One site we pruned at Atlassian gained significantly in top 20 rankings, and we gained roughly 25% in organic traffic about a month after we pruned.
# of keywords over the last 12 months in SEMrush.
How often should you prune?
I recommend 1-2x/year; otherwise, you risk “over pruning”. It is like cutting a tree too short. Sometimes, the results need a bit of time to show. It often depends on the size of the site; a larger site will show results faster.
Who should prune?
Everybody, but your site shouldn’t be “too small”. If you have 100 pages, it probably doesn’t make sense, but you can start thinking about it from 1,000 pages on. Keep in mind that these are arbitrary values that I’m providing to give a rough idea of what “small” and “large” is. You need some content to prune. You can’t cut branches off that aren’t there, yet.
What else can you prune?
You can (and should) prune backlinks and inventory after the same principle: disavow spammy backlinks and get rid of products nobody buys to improve the overall quality of your profile. It is part of keeping things in order.
Sometimes, less is more.
Comments
Provides valuable insights and adds depth to the conversation.
Either just recently joined or is too shy to say something.
Thanks again!
A bearer of digital marketing wisdom.
the method should be doable for sites with thousands of pages.
Did you choose "Landing pages / SEMrush" or the other option? It worked beyond 50 URLS for me.
Cheers,
Kevin
Either just recently joined or is too shy to say something.
A bearer of digital marketing wisdom.
> Its criteria are if a page gets less than 15 pageviews in 30 days, and the page was published 2 years or more ago.
I like that idea and think it applies to almost every site. However, for more nuanced recommendations these values have to be customized to the site.
> My site gets a lot of traffic but only has around 280 pages. It's suggesting that about 70 pages need to be rewritten or removed. A lot of them are pretty standard articles, not particularly spammy or anything. What are your thoughts on this?
Hard to say without looking at the site and data. How is the engagement for the 70 articles?
Either just recently joined or is too shy to say something.
Most of these pages have about 1-10 sessions a month. Those visits often have 0 for session duration, but some will have a few minutes. A lot of the pages have one or two backlinks from articles on industry sites or Quora, but I'm not sure if these backlinks have any value if they only send maybe one visitor a month...
A bit hair-raising culling pages as I don't want ranking to suddenly take a nosedive! lol
Any thoughts welcomed, thanks
A bearer of digital marketing wisdom.
Either just recently joined or is too shy to say something.
Either just recently joined or is too shy to say something.
Provides valuable insights and adds depth to the conversation.
When you say that big sites get results faster, did you mean to say that since they get traffic traffic, it's easier to see what happens to it after pruning?
Also, will pruning a site that gets little traffic, and that only has few pages that are weighing it down, help it pick up steam quicker?
Thanks Kevin:)
A bearer of digital marketing wisdom.
> When you say that big sites get results faster, did you mean to say that since they get traffic, it's easier to see what happens to it after pruning?
I mean that you seem to need a certain size for pruning to be effective. That could have something to do with crawl budget or another threshold Google measures that we're not aware of. If your blog has only 50 pages, for example, the effects might not be as strong as if you had 500 pages. It doesn't mean that pruning doesn't work at all, just that the effects are less significant.
> Also, will pruning a site that gets little traffic, and that only has few pages that are weighing it down, help it pick up steam quicker?
Do you mean a site that "only has a few pages" or a site that "has a few underperforming pages"?
Cheers,
Kevin
Provides valuable insights and adds depth to the conversation.
Thanks for your helpful comment.
I guessed that having a decent sized website opens it up to all potential benefits of SEO pruning, but that it also works on smaller websites.
> Do you mean a site that "only has a few pages" or a site that "has a few under-performing pages"?
I meant a site that has a few pages (not under-performers since they all are- the site is barely getting any traffic).
I ask you this because I recently listened to Brian Dean on OkDork podcast, and he said that one of the reasons Backlinko was able to compete with the giants is that he kept his site lean and that every link he got counted way more for him that it would to, for example, Moz (they have over 60 000 indexed pages)
To sum it up:
is it worth to prune a very small site while building high-quality links?
Will this concentrated link equity help the site rank faster and higher than it normally would (for a site that is not pruned, when authority is spread across both useful and dead-weight pages)?
Thanks for your reply,
Cheers!
Nikola Roza
A bearer of digital marketing wisdom.
you're welcome. I think what Brian Dean does with his site is pretty much the same pruning. He creates few but super high-quality and long-form content. I don't think there's much to prune on his site because he keeps his quality so high.
> Will this concentrated link equity help the site rank faster and higher than it normally would (for a site that is not pruned, when authority is spread across both useful and dead-weight pages)?
I think the concentration of link equity on a site with few but high-quality pages plays a role but it's not the only reason the site ranks so well. The good content is more important, I'd say.
Cheers,
Kevin
Provides valuable insights and adds depth to the conversation.
Occasionally takes part in conversations.
Thanks for sharing informative Tips for Step-By-Step Guide: SEO.
Awesome
Either just recently joined or is too shy to say something.
Either just recently joined or is too shy to say something.
Either just recently joined or is too shy to say something.
We put out quite a bit of new content every day (1000 product pages, user generated).
So far in my crawl, 80% of the pages on the site have no search console data (zero impressions).
Only ~3.5% of the pages have 1 or more click in the past 90 days.
Would you start with just pruning the bottom 80% don't even have impressions? Then get the remaining 16.5% after watching for a few weeks?
I'm sure it will take weeks for the pages to fall out of the index anyways due to the number and crawl restraints.
A bearer of digital marketing wisdom.
Once you set the posts to noindex, try putting them all in an XML sitemap and ping it to Google. don't keep it long-term, just unti Google crawled the pages.
Either just recently joined or is too shy to say something.
Either just recently joined or is too shy to say something.
A bearer of digital marketing wisdom.
There is also the lens you look through: if a recipe is underperforming in terms of traffic, it's a strong point for fully pruning it. If it's just underperforming in terms of rankings/organic traffic but gets lot's of shares, for example, then meta=noindex might be a good try. The good thing about noindexing is that it's reversible.
Hope that helps,
Kevin
Occasionally takes part in conversations.
If it was my site, I must have given it a try: before pruning or no-indexing:
- Do an effective keyword research for the underperforming recipes.
- List down the perfect keywords that you think can get you traffic.
- Re-edit the content, make them better, better than the competitors.
- Add the keywords and publish it.
Re-editing the old content can give you a ranking boost.
Asks great questions and provides brilliant answers.
A bearer of digital marketing wisdom.
Good luck,
Kevin
Either just recently joined or is too shy to say something.
A bearer of digital marketing wisdom.
Either just recently joined or is too shy to say something.
Either just recently joined or is too shy to say something.
That's an interesting read and I'm actually in the midst of content audit and pruning for [link removed by moderator]. I was doing it category wise and it required quite a manual work. Still got to learn VLOOKUP to group all data per URL.
Thanks for the guide.
A bearer of digital marketing wisdom.
Occasionally takes part in conversations.
Either just recently joined or is too shy to say something.
A bearer of digital marketing wisdom.
Provides valuable insights and adds depth to the conversation.
A bearer of digital marketing wisdom.
Occasionally takes part in conversations.
A bearer of digital marketing wisdom.