What is Meta-ExternalFetcher?
Meta-ExternalFetcher fetches individual links when a user action in a Meta product needs them. It is not a general crawler.
Last verified 2026-09-30 against the operator's documentation. All bots
Meta-ExternalFetcher at a glance
- Operator
- Meta
- Type
- AI user-triggered fetchers
- Purpose
- Fetches individual links on behalf of Meta product users.
- robots.txt token
meta-externalfetcher- How to verify
- Meta says a crawler request comes from Meta if its source IP is on the list this command returns: whois -h whois.radb.net -- '-i origin AS32934' | grep ^route. That list covers Meta's whole network (AS32934), not this crawler alone, and Meta notes the addresses change often.
- Operator documentation
- https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/
How often is Meta-ExternalFetcher impersonated?
Meta documents one check for its crawlers: the source IP must be in its network, AS32934. Our measurement does not yet apply that check, so we show no share for Meta-ExternalFetcher.
This directory measures requests between 3 July 2026 and 28 September 2026. The fake crawler report covers its own, earlier window (late June to 21 September 2026), so its figures differ from the ones here. Read it for the method.
Have a request that claims to be Meta-ExternalFetcher? Check its IP address.
Should you block Meta-ExternalFetcher?
- Allow it if
- you want the assistant to be able to read and cite your pages when someone asks about them.
- Block it if
- your content is not meant to be fetched on demand, and you enforce that at the edge.
- What blocking changes
- Meta says this fetcher may bypass robots.txt, so a Disallow line may not stop it.
To block it, add this group to your robots.txt. Meta says that because a user requested the fetch, this crawler may bypass robots.txt.
User-agent: meta-externalfetcher
Disallow: /robots.txt only asks. It does nothing against a client that ignores it or only pretends to be Meta-ExternalFetcher. Stopping those takes verification at your edge.
Meta-ExternalFetcher: common questions
See which bots reach your site, and which of them are who they claim to be.
What is Meta-ExternalFetcher?
Meta-ExternalFetcher fetches individual links when a user action in a Meta product needs them. It is not a general crawler.
How do I verify that a request is really Meta-ExternalFetcher?
Meta says a crawler request comes from Meta if its source IP is on the list this command returns: whois -h whois.radb.net -- '-i origin AS32934' | grep ^route. That list covers Meta's whole network (AS32934), not this crawler alone, and Meta notes the addresses change often. The user agent alone proves nothing: any client can send it.
How do I block Meta-ExternalFetcher in robots.txt?
Add a group for User-agent: meta-externalfetcher with Disallow: /. Meta says that because a user requested the fetch, this crawler may bypass robots.txt. robots.txt only asks. A client that ignores it, or pretends to be Meta-ExternalFetcher, has to be stopped at your edge.
What happens if I block Meta-ExternalFetcher?
Meta says this fetcher may bypass robots.txt, so a Disallow line may not stop it.
Related bots
- Meta-ExternalAgent (Meta): Crawls for training AI models or improving Meta products.
- facebookexternalhit (Meta): Builds link previews when content is shared on Meta's apps.
- ChatGPT-User (OpenAI): Visits a page when a ChatGPT or Custom GPT user's request needs it.
- Claude-User (Anthropic): Fetches a page when a Claude user's question needs it.
- Perplexity-User (Perplexity): Fetches a page when a Perplexity user's request needs it.
- MistralAI-User (Mistral AI): Fetches a page when a Mistral user's action needs it.