What is SemrushBot?
SemrushBot is Semrush's crawler. It discovers and collects new and updated web data for Semrush's tools, including its backlink index. Semrush runs several variants with their own robots.txt tokens, such as SemrushBot-BA and SiteAuditBot.
- Operator
- Semrush
- Purpose
- Collects web data for Semrush's SEO tools.
- robots.txt token
SemrushBot- How to verify
- Semrush publishes no IP list and says it does not use consecutive IP blocks, so it asks you not to block it by IP.
- Operator documentation
- https://www.semrush.com/bot/
- Last verified
How often is SemrushBot impersonated?
Our verification data for SemrushBot was incomplete during the measurement window, so a share would overstate impersonation. We show none until it is complete.
This directory measures requests between 14 July 2026 and 10 October 2026. The fake crawler report covers its own, earlier window (late June to 21 September 2026), so its figures differ from the ones here. Read it for the method.
Have a request that claims to be SemrushBot? Check its IP address.
Should you block SemrushBot?
- Allow it if
- you or your partners use this tool's backlink or audit data, or the crawl load is not a concern.
- Block it if
- the crawl load is noticeable and you have no use for the tool's data. Blocking it does not affect search rankings.
- What blocking changes
- Semrush stops collecting data about your site for its tools. Each variant needs its own robots.txt line.
To block it, add this group to your robots.txt. Semrush says SemrushBot obeys robots.txt and honors Crawl-delay up to 10 seconds.
User-agent: SemrushBot
Disallow: /robots.txt only asks. It does nothing against a client that ignores it or only pretends to be SemrushBot. Stopping those takes verification at your edge.
Related bots
- AhrefsBot (Ahrefs): Builds the link database behind Ahrefs and the Yep search engine.
- AhrefsSiteAudit (Ahrefs): Crawls a site for an Ahrefs Site Audit project.
- Barkrowler (Babbar): Collects links and page metadata for Babbar.
- DataForSeoBot (DataForSEO): Adds links to DataForSEO's backlink database.
- DotBot (Moz): Gathers data for the Moz Link Index.
- MJ12bot (Majestic): Maps links between websites for Majestic's link index.
See which bots reach your site, and which of them are who they claim to be.
Request a site audit