1592+ crawlers perfilados
Directorio de Crawlers
Todos los crawlers web y bots de IA que rastreamos. Qué hace cada uno, quién lo opera y cómo proteger su contenido.
Splunk Synthetics — What It Is and How to Handle It
Splunk Synthetics is the monitoring bot that proactively tests website performance, APIs, and user flows from global locations to detect and fix performance issues before customers are affected.
MonitoringMonitoring
SolarWinds Observability — What It Is and How to Handle It
SolarWinds Observability is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
ZoomInfo — What It Is and How to Handle It
Zoominfobot is an indexing robot for a web search engine, similar to Google. Created by Zoom Information Inc.(www.zoominfo.com), Zoominfobot’s patented technology continually scans millions of corporate websites, press releases, electronic news services, SEC filings and other online sources. Using advanced natural language processing algorithms, ZoomInfo has created a next generation search engine focused on finding pages with information about businesses and business professionals.
Search EngineSearch Engine
UpDownBot — What It Is and How to Handle It
UpDownBot is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Crazy Egg Bot — What It Is and How to Handle It
Crazy Egg Bot is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Alertsite by Smartbear — What It Is and How to Handle It
AlertSite by SmartBear monitors website and API availability, performance, and functionality from global locations as part of SmartBear's testing platform.
MonitoringMonitoring
cron-job.org — What It Is and How to Handle It
Scheduled execution of your websites and scripts.
OtherOther
NitroBot — What It Is and How to Handle It
We run a cloud based site speed optimization solution. As such, we need to make requests to our clients' sites in order to fetch the content that needs to be optimized. We have several sub systems that can fire requests and each one can be identified based on the user agent suffix.
SEO ToolSEO Tool
360Monitoring — What It Is and How to Handle It
360Monitoring checks website uptime, performance, and server health across multiple global locations, providing alerts for downtime and slow response times.
MonitoringMonitoring
Internet Archive — What It Is and How to Handle It
The Internet Archive bot, also known as archive.org_bot, is the web crawler for the Internet Archive's Wayback Machine. It systematically crawls and preserves publicly accessible web pages for historical record.
ArchiverArchiver
Exodus — What It Is and How to Handle It
A crypto wallet application to manage cryptocurrencies like Bitcoin, Ethereum, Ripple, and more. Secure.
OtherOther
Kinsta — What It Is and How to Handle It
Kinsta is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Trendiction Bot — What It Is and How to Handle It
Talkwalker delivers the consumer insights that help brands drive business impact. In a world full of conversations, the most successful global brands have switched to Talkwalker because we provide them with a powerful software platform to uncover, understand and derive the most valuable insights from internal and external data. Our listening and analytics platform enables more than 2,500 companies worldwide to protect their brands, measure their impact and gain the key consumer insights that drive purchase decisions.
MonitoringMonitoring
Pulsetic — What It Is and How to Handle It
Pulsetic is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
CCBot — What It Is and How to Handle It
CCBot crawls the web for Common Crawl, a nonprofit that provides free web archives used to train many AI models including GPT and Claude.
AI CrawlerAI Training
Dataprovider.com — What It Is and How to Handle It
Dataprovider.com collects web data for business intelligence, company research, and data enrichment services.
Search EngineSearch Engine
SE Ranking Bot — What It Is and How to Handle It
SE Ranking Bot is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Google Digital Asset Links — What It Is and How to Handle It
Google Digital Asset Links verifies app deep links and website associations for Android apps through assetlinks.json files.
SEO ToolSEO Tool
OnCrawl — What It Is and How to Handle It
OnCrawl is an enterprise SEO crawler that performs technical audits, log analysis, and data-driven SEO optimization.
SEO ToolSEO Tool
Checkly Bot — What It Is and How to Handle It
Checkly is a high-programmability active monitoring solution. We support users in monitoring their websites and APIs. Puppeteer and Playwright (both supported) are browser automation tools that can be used for a variety of tasks. For testing, they really are about E2E/component testing, not unit testing.
MonitoringMonitoring
Amazon AdBot — What It Is and How to Handle It
AmazonAdBot is the web crawler used by Amazon's advertising services to visit and analyze advertiser landing pages to ensure compliance with advertising policies and optimize ad campaign performance.
AdvertisingAdvertising
Bing Preview — What It Is and How to Handle It
Bing Preview generates page thumbnails and metadata for link previews in Bing search results, Outlook, Teams, and other Microsoft platforms.
PreviewPreview
Sogou — What It Is and How to Handle It
Sogou web spider is the crawler for Sogou, a major Chinese search engine now part of Tencent, with over 10 billion Chinese pages indexed.
Search EngineSearch Engine
SendGrid — What It Is and How to Handle It
The SendGrid Event Webhook sends email event data to customer APIs as SendGrid processes it.
OtherOther
Uptime LLC — What It Is and How to Handle It
Uptime LLC is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Grafana's Synthetic Monitoring — What It Is and How to Handle It
Grafana's Synthetic Monitoring is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Yeti — What It Is and How to Handle It
Yeti is Naver's web crawler that systematically discovers and indexes web content for South Korea's largest search engine. It crawls websites to build Naver's search index and support various platform services.
Search EngineSearch Engine
logicmonitor — What It Is and How to Handle It
logicmonitor is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
FullStory — What It Is and How to Handle It
FullStory is your digital experience analytics platform for on-the-fly funnels, pixel-perfect replay, custom events, heat maps, advanced search, Dev Tools, and more. FullStoryBot’s fetches and stores assets required to rebuild sites when viewing recorded sessions.
MonitoringMonitoring
TTD Content — What It Is and How to Handle It
TTD-Content is a web crawler that classifies and analyzes webpage content for programmatic advertising purposes, helping advertisers determine brand-safe placement opportunities and content suitability for targeted advertising campaigns.
AdvertisingAdvertising