Anyone can put "Googlebot" in a user-agent header. Between late June and 21 September 2026 we checked every request that named a well-known crawler on the news and media sites Centinel protects, using each operator's own verification method. On the typical site, about 1 in 7 requests calling itself Googlebot did not come from Google. For several AI crawlers, most requests using the name were fake, and nearly every request claiming to be Perplexity-User or Google-Extended was.
All figures below are shares, not counts. The method is at the end.
Most impersonated crawler identities
A request failed verification when its address was outside the operator's published ranges or it failed the operator's documented DNS check. "Typical site" is the median across sites. "All traffic" pools every request, so the busiest sites count for more.
| Claimed identity | Typical site | All traffic |
|---|---|---|
| Google-Extended | over 99% | over 99% |
| Perplexity-User | 99% | 96% |
| Claude-SearchBot | 99% | 2% |
| CCBot | 88% | 44% |
| GPTBot | 84% | 2% |
| Meta-ExternalAgent | 77% | 2% |
| ClaudeBot | 40% | 6% |
| Claude-User | 25% | 24% |
| PerplexityBot | 20% | 19% |
| Googlebot | 14% | 6% |
| ChatGPT-User | 12% | 1% |
| OAI-SearchBot | 12% | 4% |
| Bingbot, Applebot, Amazonbot | 1% or less | under 1% |
The columns differ because the real crawlers are unevenly spread. A few busy sites get most of the genuine GPTBot and Claude-SearchBot traffic, which pulls the pooled share down. Elsewhere the real crawler rarely visits, so the name mostly arrives on fake requests. On the typical site, most requests calling themselves GPTBot were impersonators.
Each figure comes from at least five sites, and each median uses only sites with a meaningful number of claims. Sites differ a lot: the share of fake Googlebot ranged from under 1% on one site to nearly all Googlebot claims on another.
Google-Extended is never a real visitor
Google-Extended is a robots.txt token, not a crawler. Google's crawler documentation says it "doesn't have a separate HTTP request user agent string" and that crawling uses Google's existing user agents. By Google's own description, a request that sends Google-Extended as its user agent is not from Google, so you can block it without a lookup.
Where the fakes come from
Fake Googlebot comes from hosting networks. 88% of those requests came from datacenter addresses, and about a third came from the networks of a single European hosting provider.
Fake AI crawlers mostly come from Google Cloud: at least half of the requests falsely claiming to be GPTBot, ClaudeBot, PerplexityBot or ChatGPT-User came from Google Cloud addresses. The cloud provider alone proves nothing, though. Anthropic's own published list includes Google Cloud addresses, so only the exact address separates the real crawler from the fake.
Fakes also go for articles. 95% of fake Googlebot requests asked for pages rather than images, scripts or stylesheets, against 85% for the real Googlebot.
The version string does not give fakes away
Impersonators copy the current version. 94% of fake Googlebot requests claimed the same version as the real Googlebot, and for ChatGPT-User, Bingbot, Amazonbot and Claude-SearchBot every fake used the real crawler's version. Checking the version catches nothing.
A few fakes are careless. 99% of fake CCBot requests sent the bare crawler token without the
usual Mozilla/5.0 prefix. Across all identities, though, 90% of fake requests used the full
browser-style string.
Allowing crawlers by name lets the fakes in
A common shortcut is to allow search and AI crawlers by user-agent string, so that a bot rule cannot hurt search visibility. Under that rule, every fake request counted above passes as Google, OpenAI or Anthropic and gets whatever treatment the rule gives the real crawler.
Verify first, then allow:
- Check the address against the operator's published list or its documented reverse and forward DNS check. Google, Bing, OpenAI, Anthropic and Perplexity publish machine-readable IP lists. The AI agent verification guide covers each operator's method, and the crawler verification tool checks a single address.
- Keep verified, contradicted and unknown as separate results. A contradicted claim is stronger evidence of intent than an unknown one.
- Tie the allow rule to the verified operator, not to the string. Keep your robots.txt policy for the real crawlers; the fakes do not read it.
We checked Centinel's verification against the operators' own lists for the week of 15 to 21 September. For Google, Bing, OpenAI and Perplexity, 99.99% of the requests Centinel verified came from the published ranges, and 98% of the requests it did not verify came from outside them. A one-day check against Anthropic's list agreed: every unverified Claude request came from outside it.
Automated browsers
Fake crawlers at least name themselves. Automated browsers, meaning headless or scripted browsers running real rendering engines, claim to be ordinary visitors instead. Among the requests we classified as automated browsers, 56% presented a Chrome user agent and 70% came from datacenter addresses. Their user agent cannot be checked against a published list, so detecting them takes behavioral and browser evidence. The AI agent traffic guide covers the agents in this group.
Method
Data: requests to news and media sites protected by Centinel, from late June to 21 September 2026, excluding test and staging hosts. Figures are medians across sites or pooled across all traffic, as labelled. We publish a figure only when at least five sites contributed, and no site, URL or visitor can be identified.
Claim: the user agent contains the crawler's documented token.
Verification: the operator's documented method, in this order: a signed request where the operator supports it, then published IP ranges, then reverse and forward DNS. A request fails when the method contradicts the claim. We left out two identities where our check was unreliable: Facebook's link-preview fetcher, whose failed requests came largely from home broadband networks, and Semrush, whose failed requests came from Semrush's own network.
Network labels come from Centinel's IP intelligence. "Datacenter" means the address belongs to a hosting or cloud network.
Limits: shares depend on which sites were in the pool, and a single week can differ sharply from the 13-week figure. This first edition therefore reports no trend; the next edition will compare like for like.
The crawler audit guide shows how to run the same check on your own logs.