Site Audit Crawled Pages Report Manual

The Crawled Pages section of your Site Audit lists all of the URLs that were crawled by our bot.  This gives you an easy way to look up every page on your site that was crawled, and analyze the status of your website on a page-by-page basis.

The table provides in-depth data about every crawled page, including:

  • Markups
  • Crawl depth
  • Percentage of pages with AMP (Accelerated Mobile Page) links
  • If your pages are present in the sitemap

To work through this report, you can filter the pages by URL or status (pages with issues, broken pages, redirects, blocked pages, and healthy pages). Currently, you can combine up to five advanced filters at once, but we are working on more which will be coming soon.

For example add two filters to the report to locate pages that have Open Graph markup and a crawl depth of +4 on your site (see below).

To re-crawl a specific page, click on the circular icon in the far right hand column, under “Re-audit URL.” This will send our crawler to just that one page and check for issues. This is an efficient way to follow the progress of site maintenance without using up an unnecessary amount of your crawl budget.

Site Audit Crawled Pages Report image 1

Why are these page attributes important for SEO?

Crawl depth of a page refers to the number of clicks needed to reach a specific page from the domain’s homepage using the shortest path. The homepage will have a depth of 1 and any page linked from the homepage has a depth of 2.

For the most important content on your website, it’s best practice to have a crawl depth below 3.

Similarly, it’s beneficial for your most important pages to be present in your sitemap because that makes it easier for crawlers to locate your content.

An Accelerated Mobile Page is webpage with simplified HTML that experiences faster loading speed so that users can have quicker access to the page content from a mobile device.

Using proper markups in your HTML can help search engines identify the entities on your website and index your content more accurately.

You can find more detailed information about each of these attributes in the Statistics report of your Site Audit.

Individual URL report

To see that page’s individual URL report, click on any URL in the table.

Site Audit Crawled Pages Report image 2

The next page will list the errors, warnings, and notices as well as any incoming internal links for each crawled page. You can hover over the info button for an explanation of the issue, click on each issue for more details, or click on the link in the right column to see how many other pages on the site had the same issue.

Or, you can select the blue square and arrow icon beside the URL to open up the webpage in a new tab.

Log in

By clicking on "Create my account", you are agreeing to our Terms of Use.