FREE online SEO course: On-Page and Technical SEO CourseSEO Audit7 lessons
The Importance of Technical SEO
Hi there! This is the beginning of our course, and it's devoted to technical SEO. Technical SEO is actually the foundation to your entire SEO strategy. It's all about how your website is built and structured, and how easy it is for search engines to crawl and index your content. Some technical SEO issues might even render your website invisible to search engines, so it's critical to understand, define, and be able to fix them.
How to Do a Technical SEO Audit with SEMrush
So, in the first lesson of our module, we will check on and take steps to improve your website's health. We will streamline this process with the help of our all-round technical SEO tool — Site Audit.
It helps you find, prioritize and fix technical and on-page issues to boost your website's health and your SEO.
SEMrush Site Audit Tool Setup
Let's learn more about Site Audit, and start with setting it up.
The First Step: Choosing the Crawl Scope
- Enter your crawl scope. It can be a domain, a subdomain, or a subfolder.
- Set the limit of crawled pages. You can crawl up to 100,000 pages per audit.
- Specify the crawl source. It can be the website itself, its sitemap, or even a custom sitemap. If you want to give your AMP pages priority over other pages during the crawl, check the Crawl AMP pages first checkbox below.
The Second Step: Crawler Settings
- Choose a user agent name to crawl your site. It can be either the mobile or desktop version of the SEMrush Bot or the Google Bot.
- Then set a crawl-delay. We recommend choosing the minimum delay option to maximize the auditing speed, but if you suspect that our crawler slows down your website, then choose the second option to tell it to respect the crawl-delay directive from your robots.txt file. The third option is introduced in case you don't have access to your robots.txt file, but you want to avoid your website's performance drop during crawling. If you choose this option, the crawling speed will be limited to 1 URL per 2 seconds.
The Third Step: Allow/Disallow Rules
Apply allow/disallow rules to include or exclude specific parts of your site from the audit. Entering a specific subfolder in the Allow box will narrow down the audit scope to this very subfolder. Disallow rules, in turn, tell the tool to exclude the entered subfolder from crawling.
The Fourth Step: Specifying URL Parameters
Specify URL parameters you want to be ignored during crawling. As a result, certain URLs with and without entered parameters will be considered as the same page.
The Fifth Step: Scheduling Crawls
Schedule automatic audits, setting the preferred day of the week for the audit. You can also check the checkbox above the Start Site Audit button to let the tool send you an email once an audit is complete. Click this button to start the audit.
Connecting your Google Analytics Account
Later on, you'll be able to connect your Google Analytics account. We recommend doing this for the sake of prioritization: your top-viewed pages will show up first in the audit list. Remember, you can always read just setup parameters for future recrawls.
SEMrush Site Audit Overview Report
Once the report is ready, you will see the tool's main screen. It comprises:
- The Total Score metric, which reflects the density of your website's problems.
- The crawled pages count with the breakdown of your pages by their status - healthy/broken/have issues/redirects/blocked.
- The Robots.txt Updates widget, which checks the robots.txt file for availability and changes made to it since the previous crawl. It's important to keep abreast of even minor changes to your robots.txt file, since any issue may damage your rankings.
- Three types of issues that prevent your website from getting high rankings or cause a bad user experience.
These types are grouped by their impact on your website's health in descending order: errors, warnings and notices. We advise you to focus on errors first, since those affect your SEO efforts the most.
SEMrush Site Audit Thematic Reports
Below, there's a list of thematic reports with personal scores of your website:
- HTTPS implementation
- International SEO
- Performance, and
- Internal Linking
By clicking the button of any report, you'll be brought through to a page with detailed information about the corresponding technical SEO aspect. At the bottom of the page, you'll see a list of your website's top issues. These issues are sorted by their severity.
How to Start Fixing Your Website's Technical Issues
There are several ways to start reviewing and fixing the issues. Let's click the blue button to get to the report with the list of all issues. Every issue in the list has a Why and how to fix it tooltip to give you more information about it. Clicking on a certain issue will reveal URLs with that issue. You can hide issues you've fixed, or consider irrelevant or unimportant. Be aware that hiding issues will prevent Site Audit from checking for it in future recrawls. Finally, you can send the tasks regarding the issues directly to Trello, asking your teammates to deal with them.
Another way of tackling the issues is doing it page by page under the Crawled Pages tab. You can sort your pages by number of issues or, providing you have connected your Google Analytics account, by unique pageviews. You can apply different filters to select certain pages to start working with. For instance, you may want to start fixing pages with slow load speed right away, or you can't wait to clean up the mess with your broken pages. A useful feature here is that you can re-audit a chosen URL to let the tool perform the checks right away.
Toggle the Site structure view to see the distribution of issues across your website's subfolders.
SEMrush Site Audit Statistics Report
Go to the Statistics page to see the statistical summary of your website. There you'll find out if your website's pages:
- Have no markup. In this case, you may want to implement some.
- Are more than 3 clicks away from the homepage. It would be wise to make these pages more reachable for users and crawlers.
- Have 4xx and 5xx status codes. Make sure that you have as few of these as possible.
- Are without the rel='canonical' tag. In cases where you have duplicate content on your website, you definitely need to canonicalize these pages properly to avoid jeopardizing your SEO.
- Have no AMP link. If you've implemented AMP to your website, don't forget to link your pages with the AMP version.
- Have only 1 incoming internal link. Try to furnish your pages with several incoming internal links each. Internal linking is an important part of SEO.
- Pages without hreflang issues. If you target multiple countries with different spoken languages, hreflang usage issues is something you should avoid at all costs.
After you've made your first recrawl, you'll be able to compare two crawls head-to-head in the Compare Crawls report. You'll see how your issues have changed over time. Eventually, when there are multiple crawls, you'll be free to choose any pair to compare.
SEMrush Site Audit Progress Report
To visualize the trend of your technical and on-page issue fixing endeavors, go to the Progress tab. You'll see how the number of times a certain issue was found as well as the total number of issues changed from crawl to crawl. The graph is interactive, so you can fine tune it to examine your progress on any desired issues. Then, you can add notes to track what has been done and when, and how this affected your website. The last two reports will help you to stay on top of your website's health.
Technical SEO Audit with SEMrush: Lesson Summary
So let's sum up. You can learn about your website's technical and on-page issues with the help of the Site Audit tool, which combines more than 120 checks. You can also set up weekly email notifications to stay informed of your progress. The tool is constantly evolving, so practice with it more often! Also, go to the Read Further section, refer to our Knowledge Base and take a look at our blog to learn more about Site Audit.
On-Page and Technical SEO Course
7 lessons1 hour
Learn how to improve and maintain your website's health, and optimize your website's content for both users and search engines with SEMrush tools to rank highly and gain more online visibility. Learn more.
You’ve completed the course! The course is left open for you, so you can go back and watch the videos at any time navigating through the left-hand menu. Now take the next step towards perfection and try to pass the On-Page and Technical SEO Test.
Share your achievement
You are going to start the !
Before you start, review the information below:
Certify your team request
Want to rest assured that your employees’ knowledge of the SEMrush toolset is up to the mark?
If you’re interested in making a request, please leave your email.
Become a contributor
Want to join the SEMrush Academy expert community? You will have the opportunity to reveal your own tips and tricks on SEMrush tools. If you’re interested, leave your email.