What is Meta-ExternalFetcher?
Meta-ExternalFetcher is the Meta crawler that fetches individual links at a user's request. Meta says it supports product functions such as evaluating and improving agentic AI capabilities, including helping AI navigate websites to complete tasks for users.
- Operator
- Meta
- Purpose
- Fetches individual links on behalf of Meta product users.
- robots.txt token
meta-externalfetcher- Example user agent
meta-externalfetcher/1.1- How to verify
- Meta's crawler page gives no IP list and no reverse DNS domain for Meta-ExternalFetcher.
- Operator documentation
- https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/
- Last verified
How often is Meta-ExternalFetcher impersonated?
No published method checks a request that claims to be Meta-ExternalFetcher, so we cannot say which claims are false and show no share.
This directory measures requests between 14 July 2026 and 10 October 2026. The fake crawler report covers its own, earlier window (late June to 21 September 2026), so its figures differ from the ones here. Read it for the method.
Have a request that claims to be Meta-ExternalFetcher? Check its IP address.
Should you block Meta-ExternalFetcher?
- Allow it if
- you want Meta's AI products to be able to open your pages when a user asks.
- Block it if
- your content is not meant to be fetched on demand, and you enforce that at the edge.
- What blocking changes
- Meta says this fetcher may bypass robots.txt, so a Disallow line may not stop it.
To block it, add this group to your robots.txt. Meta says that because a user requested the fetch, this crawler may bypass robots.txt.
User-agent: meta-externalfetcher
Disallow: /robots.txt only asks. It does nothing against a client that ignores it or only pretends to be Meta-ExternalFetcher. Stopping those takes verification at your edge.
Related bots
- facebookexternalhit (Meta): Builds link previews when content is shared on Meta's apps.
- Meta-ExternalAgent (Meta): Crawls for training AI models or improving Meta products.
- Meta-WebIndexer (Meta): Crawls pages to improve the search results of Meta AI.
- ChatGPT-User (OpenAI): Visits a page when a ChatGPT or Custom GPT user's request needs it.
- Claude-User (Anthropic): Fetches a page when a Claude user's question needs it.
- MistralAI-User (Mistral AI): Fetches a page when a Mistral user's action needs it.
See which bots reach your site, and which of them are who they claim to be.
Request a site audit