What is Screaming Frog SEO Spider?

How often is Screaming Frog SEO Spider impersonated?

We have not measured requests claiming to be Screaming Frog SEO Spider yet, so we show no share.

This directory measures requests between 3 July 2026 and 28 September 2026. The fake crawler report covers its own, earlier window (late June to 21 September 2026), so its figures differ from the ones here. Read it for the method.

Have a request that claims to be Screaming Frog SEO Spider? Check its IP address.

Should you block Screaming Frog SEO Spider?

Allow it if

you or your agency audit this site with the tool.

Block it if

audit crawls by other people add load you do not want. A robots.txt rule stops only the users who keep the default settings.

What blocking changes

Screaming Frog says the SEO Spider cannot crawl a site that disallows it in robots.txt. The paid version has an option to ignore robots.txt, and the operator says its use is the responsibility of the user entirely.

To block it, add this group to your robots.txt. Screaming Frog says the SEO Spider obeys robots.txt by default. When no group names it, it follows the rules for Googlebot, then the rules for all robots. A user of the paid version can set it to ignore robots.txt.

User-agent: Screaming Frog SEO Spider
Disallow: /

robots.txt only asks. It does nothing against a client that ignores it or only pretends to be Screaming Frog SEO Spider. Stopping those takes verification at your edge.