1592+ Crawler profiliert
Crawler-Verzeichnis
Alle Web-Crawler und KI-Bots, die wir verfolgen. Was sie tun, wer sie betreibt und wie Sie Ihre Inhalte davor schützen.
CheeseBot — What It Is and How to Handle It
CheeseBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
CheckMarkNetwork — What It Is and How to Handle It
CheckMarkNetwork is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Checkly — What It Is and How to Handle It
Checkly is a synthetic monitoring platform that performs automated end-to-end testing and API monitoring using Playwright and Puppeteer to check website functionality, performance, and uptime across multiple locations.
MonitoringMonitoring
ChatGPT Agent — What It Is and How to Handle It
ChatGPT Agent is an AI agent created by OpenAI that can use a web browser. It can intelligently navigate and interact with websites to complete multi-step tasks on behalf of a human user.
AI CrawlerAI Assistant
ChatGLM-Spider — What It Is and How to Handle It
ChatGLM-Spider is a web crawler operated by Zhipu AI, the Chinese company behind ChatGLM. It is used for collecting data to train and evaluate the company's large language models.
AI CrawlerAI Training
Channel3Bot — What It Is and How to Handle It
Channel3Bot is Channel3's web crawler that visits publicly accessible product detail pages to index product information into a universal product catalog. The bot collects product details such as images, titles, descriptions, prices, and availability to enable AI apps and agents to discover products and direct shoppers to the original websites.
AI SearchAI Search
channable — What It Is and How to Handle It
channable bot automates multichannel e-commerce marketing by crawling product data from webshops to optimize feeds, listings, and advertising campaigns across 3000+ platforms.
MonitoringMonitoring
changedetection — What It Is and How to Handle It
changedetection is an open-source website change monitoring tool that tracks content changes on web pages.
MonitoringMonitoring
cf-qualys-scanner — What It Is and How to Handle It
cf-qualys-scanner is a cloud-based web application security scanner that provides automated crawling and vulnerability testing to identify security flaws including XSS, SQL injection, and OWASP Top 10 vulnerabilities.
SecuritySecurity
CertChief — What It Is and How to Handle It
CertChief is a certificate monitoring bot that periodically crawls web properties to check SSL/TLS certificate configurations, expiration dates, and security status.
SecuritySecurity
Centurybot — What It Is and How to Handle It
Centurybot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
CentComBot — What It Is and How to Handle It
CentComBot is an uncategorized agent operated by MelonNET. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
CensysInspect — What It Is and How to Handle It
CensysInspect is a security scanner operated by Censys that scans and indexes internet-connected devices and services. The bot systematically crawls websites and servers to collect information about internet infrastructure, including details about certificates, protocols, and configurations.
SecuritySecurity
Cegbfeieh — What It Is and How to Handle It
Cegbfeieh is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
CDSCbot — What It Is and How to Handle It
CDSCbot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
cc_bot — What It Is and How to Handle It
cc_bot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
CC Metadata Scaper — What It Is and How to Handle It
CC Metadata Scaper extracts Creative Commons metadata from web pages to catalog openly licensed content.
MonitoringMonitoring
caveman-hunter — What It Is and How to Handle It
caveman-hunter is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
careerbot — What It Is and How to Handle It
careerbot is a job aggregator crawler that indexes career and job posting pages.
AggregatorAggregator
carbon-umbrella-bot — What It Is and How to Handle It
carbon-umbrella-bot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
captify-crawler — What It Is and How to Handle It
Captify's web crawler used for collecting search intelligence data to power advertising targeting and audience insights.
SEO ToolSEO Tool
CapsuleChecker — What It Is and How to Handle It
CapsuleChecker is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
CapitalOneBot — What It Is and How to Handle It
CapitalOneBot is a web crawler used by Capital One to crawl dealer websites and gather usage information for Capital One's lead navigator button integration. It monitors how the financial services tools are being implemented and used across partner sites.
Feed ReaderFeed Reader
CandidateCrawler — What It Is and How to Handle It
CandidateCrawler is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
Caliperbot — What It Is and How to Handle It
Caliperbot is a web crawler that analyzes website content and technical features for Conductor's SEO intelligence platform, extracting HTML elements and data to power content analytics and optimization recommendations.
SEO ToolSEO Tool
Bytespider — What It Is and How to Handle It
Bytespider is a web crawler operated by ByteDance, the Chinese owner of TikTok. It's allegedly used to download training data for its LLMs (Large Language Model) including those powering ChatGPT competitor Doubao.
AI CrawlerAI Training
buzzbot — What It Is and How to Handle It
buzzbot is a news and content discovery crawler that indexes trending content.
AggregatorAggregator
Buttondown — What It Is and How to Handle It
Buttondown is an automated RSS feed crawler that fetches content from blogs and websites to convert them into newsletter content, allowing users to turn their RSS feeds into automated mailing lists.
Feed ReaderFeed Reader
BunnySlippers — What It Is and How to Handle It
BunnySlippers is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Bullseye — What It Is and How to Handle It
Bullseye is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther