How long does it take to crawl a website? It appears that my audit is stuck.Question
The amount of time it takes for our bot to crawl a website varies depending on your crawl-delay setting and the size of your audit (how many pages we need to crawl).
In some cases, you can experience a delay in the Site Audit after the crawling pages counter stops. Usually, this happens in the last stage of the audit. At this time Site Audit bot checks on the external links and images and saves the information. If your website has a lot of external links and images, it is not unusual to see a delay, and it can take some time for your audit to finish.
Most of the time, there is nothing to worry about, and you can stop crawling yourself if needed:
If you stop and save results in the middle of a crawl, the audit will present the new results of its unfinished crawl as your up-to-date audit.
If you discard results, Site Audit will not crawl or audit your pages and your crawl budget will be saved.
However, if your audit is stuck for more than 24 hours, and you cannot stop it yourself, please contact our Support team.
- How is Site Health Score calculated in the Site Audit tool?
- Why does Semrush say I have duplicate content?
- How many pages can I crawl in a Site Audit?
- How do I audit a subdomain?
- What Issues Can Site Audit Identify?
- Why does Semrush say I have an incorrect certificate?
- What are unoptimized anchors and how does Site Audit identify them?
- What do the Structured Data Markup Items in Site Audit Mean?
- Why do working pages on my website appear as broken?
- Can I change automatic Audit re-run schedule or stop it completely?
- Why are only a few of my website’s pages being crawled?
- Why can’t I find URLs from the Audit report on my website?
- How long does it take to crawl a website? It appears that my audit is stuck.
- How Does Site Audit Select Pages to Analyze for Core Web Vitals?
- Why is there a difference between GSC and Semrush Core Web Vitals data?
- Can I set up a custom re-crawl schedule?
- Configuring Site Audit
- Site Audit Overview Report
- Site Audit Thematic Reports
- Reviewing Your Site Audit Issues
- Site Audit Crawled Pages Report
- Site Audit Statistics
- Compare Crawls and Progress
- Troubleshooting Site Audit
- Exporting Site Audit Results
- How to Optimize your Site Audit Crawl Speed
- How To Integrate Site Audit with Zapier