Knowledge Base
SEMrush Reports & Tools
SEO Toolkit
On Page and Technical SEO
Site Audit
Why Did I Receive An “Auditing Domain Has Failed” Message?

Why Did I Receive An “Auditing Domain Has Failed” Message? Question

If you receive this message, the robots.txt of the domain you are trying to audit is likely disallowing user agents from crawling the site. To check the robots.txt file, enter the root domain of your site, followed by /robots.txt.

For example, the robots.txt file on target.com is found at http://www.target.com/robots.txt  

If this is the case, contact the webmaster to allow the SEMrushBot to crawl the site and try again to run the Site Audit.

Log in
Register

By clicking on "Create my account", you are agreeing to our Terms of Use.