1592+ crawlers perfilados
Directorio de Crawlers
Todos los crawlers web y bots de IA que rastreamos. Qué hace cada uno, quién lo opera y cómo proteger su contenido.
Gigabot — What It Is and How to Handle It
Gigabot is the web crawler used by Gigablast, an independent U.S. search engine that crawls and indexes web content using its own proprietary search algorithms and database.
Search EngineSearch Engine
gigablast — What It Is and How to Handle It
gigablast is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
GetRight — What It Is and How to Handle It
GetRight is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
GenomeCrawlerd — What It Is and How to Handle It
GenomeCrawlerd is a research crawler that collects genomic and bioinformatics data from scientific sources.
ResearchResearch
Genieo — What It Is and How to Handle It
Genieo is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
GeedoProductSearch — What It Is and How to Handle It
GeedoProductSearch is a crawler operated by Geedo, a product search engine that indexes billions of products from millions of online stores worldwide. The bot visits e-commerce websites to collect product information for inclusion in Geedo's search index, allowing users to search and compare products across different online retailers.
Search EngineSearch Engine
GarlikCrawler — What It Is and How to Handle It
GarlikCrawler is a security crawler from Garlik (now Experian) that monitors for identity theft and data exposure.
SecuritySecurity
gammaSpider — What It Is and How to Handle It
Focused web crawler that selectively crawls topic-specific pages
OtherOther
Gaisbot — What It Is and How to Handle It
Gaisbot is the crawler for Gais, a search engine focused on specific content domains.
Search EngineSearch Engine
g2reader-bot — What It Is and How to Handle It
g2reader-bot is the crawler for G2Reader, an RSS feed reader service.
Feed ReaderFeed Reader
G2 Web Services — What It Is and How to Handle It
G2 Web Services is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
g00g1e.net — What It Is and How to Handle It
g00g1e.net is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Fyrebot — What It Is and How to Handle It
Fyrebot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
FyndSearchEngine-ReCrawler — What It Is and How to Handle It
FyndSearchEngine-ReCrawler is a web crawler operated by fynd, a search engine service. This bot specifically handles re-crawling operations to update fynd's search index with fresh content from previously indexed websites.
Search EngineSearch Engine
FyndSearchEngine-Crawler — What It Is and How to Handle It
FyndSearchEngine-Crawler is a web crawler associated with fynd, a search engine service. This bot visits websites to index content for inclusion in the fynd search engine's database.
Search EngineSearch Engine
FullStoryBot — What It Is and How to Handle It
FullStoryBot is a web crawler that fetches and stores website assets required to rebuild user session recordings for FullStory's digital experience analytics platform, enabling pixel-perfect replay and analysis.
MonitoringMonitoring
fuelbot — What It Is and How to Handle It
fuelbot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
FriendlyCrawler — What It Is and How to Handle It
FriendlyCrawler web crawler
OtherOther
Friendica — What It Is and How to Handle It
Friendica is a feed reader that fetches and processes RSS, Atom, and other content feeds.
Feed ReaderFeed Reader
FreshRSS — What It Is and How to Handle It
FreshRSS is a feed reader that fetches and processes RSS, Atom, and other content feeds.
Feed ReaderFeed Reader
FreshpingBot — What It Is and How to Handle It
FreshpingBot is the crawler for Freshping, a website uptime monitoring service by Freshworks.
MonitoringMonitoring
Freshbot — What It Is and How to Handle It
Freshbot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
FreeWebMonitoring SiteChecker — What It Is and How to Handle It
FreeWebMonitoring SiteChecker monitors website uptime and availability for FreeWebMonitoring's free monitoring service.
MonitoringMonitoring
FreespokeCrawler — What It Is and How to Handle It
FreespokeCrawler is a search engine crawler operated by Freespoke. It visits websites to index and collect content for inclusion in Freespoke's search engine results.
Search EngineSearch Engine
fr-crawler — What It Is and How to Handle It
fr-crawler is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Foregenix — What It Is and How to Handle It
Foregenix security scanner performs passive vulnerability assessments and malware detection on eCommerce websites for banks and card brands to identify compromised merchants and prevent fraud.
SecuritySecurity
Foobot — What It Is and How to Handle It
Foobot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Flyriverbot — What It Is and How to Handle It
Flyriverbot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
fluid — What It Is and How to Handle It
fluid is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
fluffy — What It Is and How to Handle It
fluffy is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther