Anyone can put "Googlebot" in a user-agent header. From late June to 21 September 2026 we verified every request that named a well-known crawler on the news and media sites Centinel protects. On the typical site, 1 in 7 requests calling itself Googlebot did not come from Google. For several AI crawlers, most requests using the name were fake. To check one request yourself, see how to spot a fake Googlebot.
The report in four numbers1 in 7requests claiming to be Googlebot were fake on the typical site84%of GPTBot claims were fake on the typical site99%of Perplexity-User claims were fake on the typical site94%of fake Googlebot requests copied the real version stringNews and media sites protected by Centinel, late June to 21 September. The typical site is the median across sites.
Most impersonated crawler identities
A request is fake when the operator's published IP ranges or DNS check contradict it. "Typical site" is the median across sites. "All traffic" pools every request, so the busiest sites weigh more.
Share of crawler claims that failed verificationTypical siteAll trafficGoogle-Extended>99%>99%Perplexity-User99%96%Claude-SearchBot99%2%CCBot88%44%GPTBot84%2%Meta-ExternalAgent77%2%ClaudeBot40%6%Claude-User25%24%PerplexityBot20%19%Googlebot14%6%ChatGPT-User12%1%OAI-SearchBot12%4%Bingbot, Applebot, Amazonbot≤1%<1%Typical site is the median across sites with a meaningful number of claims. All traffic pools every request, so the busiest sites count for more. Each figure comes from at least five sites.
The two measures differ because real crawler traffic concentrates on a few busy sites. Those sites pull the pooled share down. On the typical site the real crawler rarely visits, so most requests using its name are fake. For GPTBot, that is 84%.
Google-Extended is never a real visitor
Google-Extended is a robots.txt token, not a crawler. Google's crawler documentation says it "doesn't have a separate HTTP request user agent string". Any request that sends it as a user agent is fake and can be blocked without a lookup.
Where the fakes come from
88% of fake Googlebot requests came from datacenter addresses, about a third of them from a single European hosting provider. At least half of the fake GPTBot, ClaudeBot, PerplexityBot and ChatGPT-User requests came from Google Cloud.
Share of fake requests sent from datacenter addressesPerplexityBot99%GPTBot98%ClaudeBot98%Claude-SearchBot98%CCBot98%Perplexity-User97%ChatGPT-User96%OAI-SearchBot96%Googlebot88%Claude-User80%Meta-ExternalAgent63%Requests that failed verification, pooled across sites. Datacenter means the address belongs to a hosting or cloud network.
Fakes target articles. 95% of fake Googlebot requests asked for pages rather than images, scripts or stylesheets, against 85% for the real Googlebot.
The version string does not give fakes away
94% of fake Googlebot requests claimed the real Googlebot's current version. For ChatGPT-User, Bingbot, Amazonbot and Claude-SearchBot, every fake did.
Fake requests that claimed the real crawler's current versionChatGPT-User>99%Bingbot>99%Amazonbot>99%Claude-SearchBot>99%Googlebot94%A version check catches almost none of them.
CCBot is the exception: 99% of its fakes sent the bare crawler token without the usual
Mozilla/5.0 prefix. Across all identities, 90% of fake requests used the full
browser-style string.
Allowing crawlers by name lets the fakes in
A common bot rule allows search and AI crawlers by user-agent string, so that blocking never costs search visibility. Under that rule, every fake request in this report gets the same treatment as the real crawler.
Verify first, then allowA request says Googlebot in its user agentCheck the address against the operator's published IP list, or run its reverse and forward DNS checkVerifiedThe address matches the operator. Apply your policy for that crawler.ContradictedThe operator's method rules the claim out. This is an impersonator.UnknownThe operator publishes no method. Treat it as an unverified bot.Keep the three results apart. A contradicted claim is stronger evidence of intent than an unknown one.
- Check the address against the operator's published IP list or its reverse and forward DNS check. Google, Bing, OpenAI, Anthropic and Perplexity publish machine-readable lists. The AI agent verification guide covers each operator's method, and the crawler verification tool checks a single address.
- Keep verified, contradicted and unknown as separate results.
- Tie the allow rule to the verified operator, not to the string. Keep your robots.txt policy for the real crawlers.
For 15 to 21 September, we compared Centinel's verification with the operators' own lists for Google, Bing, OpenAI and Perplexity. 99.99% of verified requests came from the published ranges, and 98% of unverified requests came from outside them. Against Anthropic's list, every unverified Claude request came from outside it.
Centinel's verification against the operators' own IP listsVerified requests inside the published ranges99.99%Unverified requests outside the published ranges98%Google, Bing, OpenAI and Perplexity, 15 to 21 September. A one-day check against Anthropic's list found every unverified Claude request outside it.
Automated browsers
Automated browsers are headless or scripted browsers running real rendering engines. They don't name a crawler; they claim to be ordinary visitors. 56% claimed to be Chrome and 70% came from datacenter addresses. No published list covers them, so catching them takes behavioral and browser evidence. The AI agent traffic guide covers the agents in this group.
Which browser automated browsers claimed to beChrome56%No browser identified30%Firefox6%Safari4%Edge3%Other1%Requests classified as automated browsers, pooled across sites. 70% of them came from datacenter addresses.
Method
Data: requests to news and media sites protected by Centinel, late June to 21 September 2026, excluding test and staging hosts. Each figure is a median across sites or a share of all traffic, as labelled, and comes from at least five sites.
Claim: the user agent contains the crawler's documented token.
Verification: the operator's documented method, in this order: a signed request where supported, published IP ranges, then reverse and forward DNS. A request fails when the method contradicts the claim. Facebook's link-preview fetcher and Semrush are excluded because their checks were unreliable.
Networks: "datacenter" means the address belongs to a hosting or cloud network, per Centinel's IP intelligence.
The crawler audit guide shows how to run the same check on your own logs.
Data and citation
Download the shares behind every chart (CSV). The charts and figures may be reused with this attribution:
Centinel Analytica, "Fake Googlebot and AI crawlers: 2026 verification data", September 2026, https://www.centinelanalytica.com/blog/fake-crawler-report
Until 30 September 2026 this report was titled "Fake crawlers: how often Googlebot and AI bots are not who they claim". Citations made under that title refer to the same report and data.