Your browser is out of date. The site might not be displayed correctly. Please update your browser.

Knowledge Base
Knowledge Base
Semrush Toolkits
SEO Toolkit
On Page SEO Checker
Why did I get a “page is not accessible” note for some of my pages?
What is Semrush? Getting Started How-To Articles Semrush Toolkits
SEO Toolkit
Domain Overview Organic Research Keyword Gap Backlink Gap Keyword Overview Keyword Magic Tool Keyword Manager Organic Traffic Insights Backlink Analytics Backlink Audit Link Building Tool Bulk Backlink Analysis Position Tracking Site Audit On Page SEO Checker
Log File Analyzer Listing Management Semrush Rank
Advertising Toolkit Social Media Toolkit Content Marketing Toolkit Competitive Research Toolkit Management Toolkit
SEO Glossary Semrush Integrations Subscriptions Billing and Account API Contact Us Quick Links Specialty Products Semrush Data & Metrics Semrush App Center
Recently viewed

Why did I get a “page is not accessible” note for some of my pages?

Question

This message will appear if the On Page SEO Checker crawler was blocked or unable to crawl your page. Please check the robots.txt file on your website to make sure that is allows our user agents to crawl its pages.

If our bots are not blocked in robots.txt, to fix the issue you need to whitelist the following IP addresses and User-agent with your hosting and any plugins/services you may manage your site with (i.e Cloudflare, ModSecurity):

85.208.98.53
85.208.98.0/24
User-agent: SemrushBot-SI

To specify the Port, use one of the following options:

Port 80: HTTP
Port 443: HTTPS

Additionally, you should also whitelist the Site Audit IP which is used to crawl pages at the following IP address: 

46.229.173.67
46.229.173.66
46.229.173.68
User-agent name: SemrushBot-SA

Back to Semrush Toolkits category
Frequently asked questions
Show more
Manual
Show more
Workflows
Show more