What is Googlebot?
Googlebot is Google's main search crawler. Google says rules addressed to Googlebot affect Google Search, including Discover and all Search features, and other products such as Google Images, Google Video and Google News.
Last verified 2026-09-30 against the operator's documentation. All bots
Googlebot at a glance
- Operator
- Type
- Search engine crawlers
- Purpose
- Crawls pages for Google Search, Images, Video, News and Discover.
- robots.txt token
Googlebot- Example user agent
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)- How to verify
- Run a reverse DNS lookup on the source IP and check that the name ends in googlebot.com (Google uses crawl-….googlebot.com and geo-crawl-….geo.googlebot.com), then run a forward lookup on that name and confirm it returns the same IP. Or check the IP against Google's common crawler ranges. A googleusercontent.com name does not count: any Google Cloud virtual machine can have one.
- Reverse DNS
- googlebot.com
- Operator documentation
- https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
How often is Googlebot impersonated?
For the median organisation, 13% of requests claiming to be Googlebot failed verification.
Measured between 3 July 2026 and 28 September 2026 across organisations protected by Centinel. We only publish a share when at least five organisations each saw enough of these requests. Pooled across all of their requests, the share was 6%: a few large organisations carry most of the traffic, so the median organisation is the better guide to what yours will see.
This directory measures requests between 3 July 2026 and 28 September 2026. The fake crawler report covers its own, earlier window (late June to 21 September 2026), so its figures differ from the ones here. Read it for the method.
Have a request that claims to be Googlebot? Check its IP address.
Should you block Googlebot?
- Allow it if
- you want to appear in Google Search. This is nearly every public site.
- Block it if
- a section should never appear in Google, such as internal search results. Block the path, not the crawler.
- What blocking changes
- Your pages drop out of Google Search and the other products Googlebot feeds. For almost every public site that is not the goal.
To block it, add this group to your robots.txt. Google says its common crawlers always obey robots.txt when crawling automatically.
User-agent: Googlebot
Disallow: /robots.txt only asks. It does nothing against a client that ignores it or only pretends to be Googlebot. Stopping those takes verification at your edge.
Googlebot: common questions
See which bots reach your site, and which of them are who they claim to be.
What is Googlebot?
Googlebot is Google's main search crawler. Google says rules addressed to Googlebot affect Google Search, including Discover and all Search features, and other products such as Google Images, Google Video and Google News.
How do I verify that a request is really Googlebot?
Run a reverse DNS lookup on the source IP and check that the name ends in googlebot.com (Google uses crawl-….googlebot.com and geo-crawl-….geo.googlebot.com), then run a forward lookup on that name and confirm it returns the same IP. Or check the IP against Google's common crawler ranges. A googleusercontent.com name does not count: any Google Cloud virtual machine can have one. The user agent alone proves nothing: any client can send it.
How do I block Googlebot in robots.txt?
Add a group for User-agent: Googlebot with Disallow: /. Google says its common crawlers always obey robots.txt when crawling automatically. robots.txt only asks. A client that ignores it, or pretends to be Googlebot, has to be stopped at your edge.
What happens if I block Googlebot?
Your pages drop out of Google Search and the other products Googlebot feeds. For almost every public site that is not the goal.
Related bots
- Googlebot-Image (Google): Crawls images for Google Images and image features in Search.
- Google-Extended (Google): A robots.txt token that controls Gemini training and grounding use.
- Storebot-Google (Google): Crawls product and store pages for Google Shopping.
- AdsBot-Google (Google): Checks the quality of ad landing pages for Google Ads.
- Google-InspectionTool (Google): Fetches pages for Search Console URL inspection and the Rich Results Test.
- Google-CloudVertexBot (Google): Crawls sites whose owners asked for it when building Vertex AI Agents.