How long does it take to crawl a website? It appears that my audit is stuck.Question
The amount of time it takes for our bot to crawl a website varies depending on your crawl-delay setting and the size of your audit (how many pages we need to crawl).
In some cases, you can experience a delay in the Site Audit after the crawling pages counter stops. Usually, this happens in the last stage of the audit. At this time Site Audit bot checks on the external links and images and saves the information. If your website has a lot of external links and images, it is not unusual to see a delay, and it can take some time for your audit to finish.
Most of the time, there is nothing to worry about, and you can stop crawling yourself if needed:
If you stop and save results in the middle of a crawl, the audit will present the new results of its unfinished crawl as your up-to-date audit.
If you discard results, Site Audit will not crawl or audit your pages and your crawl budget will be saved.
However, if your audit is stuck for more than 24 hours, and you cannot stop it yourself, please contact our Support team.
- What Issues Can Site Audit Identify?
- How many pages can I crawl in a Site Audit?
- How long does it take to crawl a website? It appears that my audit is stuck.
- How do I audit a subdomain?
- Can I change automatic Audit re-run schedule or stop it completely?
- Can I set up a custom re-crawl schedule?
- How is Site Health Score calculated in the Site Audit tool?
- How Does Site Audit Select Pages to Analyze for Core Web Vitals?
- How do you collect data to measure Core Web Vitals in Site Audit?
- Why is there a difference between GSC and Semrush Core Web Vitals data?
- Why are only a few of my website’s pages being crawled?
- Why do working pages on my website appear as broken?
- Why can’t I find URLs from the Audit report on my website?
- Why does Semrush say I have duplicate content?
- Why does Semrush say I have an incorrect certificate?
- What are unoptimized anchors and how does Site Audit identify them?
- What do the Structured Data Markup Items in Site Audit Mean?
- Configuring Site Audit
- Troubleshooting Site Audit
- Site Audit Overview Report
- Site Audit Thematic Reports
- Reviewing Your Site Audit Issues
- Site Audit Crawled Pages Report
- Site Audit Statistics
- Compare Crawls and Progress
- Exporting Site Audit Results
- How to Optimize your Site Audit Crawl Speed
- How To Integrate Site Audit with Zapier