1592+ Crawler profiliert
Crawler-Verzeichnis
Alle Web-Crawler und KI-Bots, die wir verfolgen. Was sie tun, wer sie betreibt und wie Sie Ihre Inhalte davor schützen.
dbot — What It Is and How to Handle It
dbot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
Dazzle BlueSky Bot — What It Is and How to Handle It
Dazzle BlueSky Bot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
Daumoa — What It Is and How to Handle It
Daumoa is the crawler for Daum, a major South Korean web portal and search engine.
Search EngineSearch Engine
Datenbank Crawler — What It Is and How to Handle It
Datenbank Crawler is a web crawler operated by German company netEstate used for collecting and selling international website data.
AI CrawlerAI Training
Dataprovider — What It Is and How to Handle It
Dataprovider is a web crawler that indexes over 700 million domains to create a structured database of website information, technologies, and business insights for data intelligence purposes.
Search EngineSearch Engine
Datanyze — What It Is and How to Handle It
Datanyze is a technographics crawler that identifies technologies used by websites for sales intelligence.
OtherOther
datagnionbot — What It Is and How to Handle It
datagnionbot is the crawler for Datagnion, a business data intelligence platform.
AggregatorAggregator
DataForSeoBot — What It Is and How to Handle It
DataForSeoBot powers DataForSEO's Backlinks API by crawling websites to analyze backlink data, providing comprehensive link analytics for SEO professionals and website optimization.
SEO ToolSEO Tool
Datafeedwatch — What It Is and How to Handle It
Datafeedwatch is a feed management crawler that helps e-commerce businesses optimize product data feeds for marketplaces.
Feed ReaderFeed Reader
Datadog — What It Is and How to Handle It
Datadog monitors website and API performance as part of its cloud monitoring platform, providing APM, infrastructure monitoring, and log management.
MonitoringMonitoring
Dark Visitor Server — What It Is and How to Handle It
Dark Visitor Server is a monitoring crawler that checks website availability, performance, and health.
MonitoringMonitoring
Dark Visitor Client — What It Is and How to Handle It
Dark Visitor Client is a headless browser crawler used for testing and research by Dark Visitors, a service that tracks AI and bot activity.
AI CrawlerAI Assistant
Dark Visitor — What It Is and How to Handle It
Dark Visitor is a web crawler operated by Dark Visitors, designed to collect data from various online sources for analysis and reporting.
MonitoringMonitoring
DareBoost — What It Is and How to Handle It
DareBoost is a website performance and quality analysis crawler that audits page speed, SEO, and security best practices.
MonitoringMonitoring
CyotekWebCopy — What It Is and How to Handle It
CyotekWebCopy is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
CydralSpider — What It Is and How to Handle It
French image search engine crawler that respects robots.txt
Search EngineSearch Engine
CyberPatrol — What It Is and How to Handle It
CyberPatrol is a web filtering crawler that categorizes websites for parental controls and content filtering.
SecuritySecurity
CyberFindCrawler — What It Is and How to Handle It
CyberFindCrawler is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
CybaaBot — What It Is and How to Handle It
CybaaBot is an uncategorized agent operated by Cybaa. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
cXensebot — What It Is and How to Handle It
cXensebot is Cxense's web crawler that performs content indexing and behavioral data collection to power Piano's customer data platform, enabling real-time analytics and audience segmentation for digital businesses.
Search EngineSearch Engine
Cutbot — What It Is and How to Handle It
Cutbot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Curebot — What It Is and How to Handle It
Curebot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
CrystalSemanticsBot — What It Is and How to Handle It
CrystalSemanticsBot is a contextual advertising crawler that analyzes web content for semantic ad targeting.
AdvertisingAdvertising
CrunchBot — What It Is and How to Handle It
CrunchBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Criticalcss.com — What It Is and How to Handle It
Criticalcss.com bot crawls websites to generate and validate critical CSS for faster loading times. The service helps developers improve website performance by optimizing above-the-fold content delivery.
MonitoringMonitoring
Crescent Internet ToolPak HTTP OLE Control v.1.0 — What It Is and How to Handle It
Crescent Internet ToolPak is a legacy ActiveX web control used for HTTP requests in Visual Basic applications.
OtherOther
Crazy Egg — What It Is and How to Handle It
Crazy Egg is a web analytics bot that takes screenshots of pages, collects assets, and tests script installation to help optimize website user experience.
MonitoringMonitoring
CrawlyProjectCrawler — What It Is and How to Handle It
CrawlyProjectCrawler is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Crawlspace — What It Is and How to Handle It
Crawlspace web crawler
OtherOther
Crawlson — What It Is and How to Handle It
Crawlson is a web crawler bot that indexes content for the Crawlson search engine by systematically browsing and analyzing web pages.
Search EngineSearch Engine