1592+ Crawler profiliert
Crawler-Verzeichnis
Alle Web-Crawler und KI-Bots, die wir verfolgen. Was sie tun, wer sie betreibt und wie Sie Ihre Inhalte davor schützen.
Blekkobot — What It Is and How to Handle It
Blekkobot was the crawler for Blekko, an independent search engine known for "slashtags" that allowed filtered searches. Blekko was acquired by IBM in 2015.
Search EngineSearch Engine
Blackboard — What It Is and How to Handle It
Blackboard (now Anthology) crawler checks links and content within learning management system courses to ensure educational resources remain accessible to students.
MonitoringMonitoring
Black Hole — What It Is and How to Handle It
Black Hole is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
bl.uk_lddc_bot — What It Is and How to Handle It
bl.uk_lddc_bot is operated by the British Library as part of their legal deposit web archiving program, which collects and preserves UK web content for the national archive.
ArchiverArchiver
BitSightBot — What It Is and How to Handle It
BitSightBot scans websites for security vulnerabilities and configurations to generate cybersecurity ratings (250-900 scale) used for third-party risk assessment.
MonitoringMonitoring
bitlybot — What It Is and How to Handle It
bitlybot verifies shortened URLs and collects page metadata (titles, descriptions) for Bitly's link management platform, ensuring links are active and safe.
Feed ReaderFeed Reader
BitBot — What It Is and How to Handle It
BitBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Birdcrawlerbot — What It Is and How to Handle It
Birdcrawlerbot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
binlar — What It Is and How to Handle It
binlar is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
BingPreview — What It Is and How to Handle It
BingPreview generates page snapshots and previews for Bing, creating visual representations of web pages for search results using both desktop and mobile variants.
PreviewPreview
BigUpDataBot — What It Is and How to Handle It
BigUpDataBot automates the pipeline of active product deals for advertisement purposes, systematically collecting product pricing and promotional data to support data-driven marketing and advertising strategies.
AggregatorAggregator
BIGLOTRON — What It Is and How to Handle It
BIGLOTRON is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
bidswitchbot — What It Is and How to Handle It
bidswitchbot is a programmatic advertising crawler operated by IPONWEB for real-time bidding ad optimization.
AdvertisingAdvertising
Better Uptime Bot — What It Is and How to Handle It
Better Uptime Bot is Better Stack's monitoring bot that checks website and infrastructure uptime, performing automated health checks and status monitoring to alert users about service outages and performance issues.
MonitoringMonitoring
betaBot — What It Is and How to Handle It
betaBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
BestChange — What It Is and How to Handle It
BestChange is the web crawler bot that continuously monitors exchange rates from hundreds of cryptocurrency and e-currency exchangers, updating rate information every 5-8 seconds to provide real-time comparison data for crypto, fiat, and digital currency exchanges.
Feed ReaderFeed Reader
BehloolBot — What It Is and How to Handle It
BehloolBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
bedrockbot — What It Is and How to Handle It
Amazon Bedrock AI crawler
AI CrawlerAI Agent
BDCbot — What It Is and How to Handle It
BDCbot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Baiduspider-video — What It Is and How to Handle It
Baiduspider-video is Baidu's specialized crawler for indexing video metadata for Baidu Video Search in China.
Search EngineSearch Engine
Baiduspider-render — What It Is and How to Handle It
Baiduspider-render is a web crawler operated by Baidu, China's largest search engine, specifically designed to render and index JavaScript-heavy pages. This variant processes dynamic web content to help Baidu understand and rank modern websites that rely on client-side rendering.
Search EngineSearch Engine
Baiduspider-news — What It Is and How to Handle It
Baiduspider-news is Baidu's specialized crawler for indexing news content for Baidu News Search in China.
Search EngineSearch Engine
Baiduspider-image — What It Is and How to Handle It
Baiduspider-image is Baidu's specialized crawler for indexing images for Baidu Image Search, essential for image visibility in the Chinese market.
Search EngineSearch Engine
Baidu-YunGuanCe — What It Is and How to Handle It
Baidu-YunGuanCe is Baidu's cloud monitoring service crawler that checks website availability and performance.
MonitoringMonitoring
Baidu-ADS — What It Is and How to Handle It
Baidu-ADS is Baidu's Anti-DDoS Service proxy that filters and scrubs web traffic during detected DDoS attacks, routing sanitized requests back to target servers for protection.
SecuritySecurity
BacklinksExtendedBot — What It Is and How to Handle It
BacklinksExtendedBot is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
BacklinkCrawler — What It Is and How to Handle It
BacklinkCrawler analyzes websites to discover and monitor backlink profiles for SEO purposes.
SEO ToolSEO Tool
backlink-check.de — What It Is and How to Handle It
German backlink checking crawler for SEO analysis
SEO ToolSEO Tool
BackDoorBot — What It Is and How to Handle It
BackDoorBot is a known malicious crawler associated with hacking attempts and unauthorized access.
ScraperScraper
b2w — What It Is and How to Handle It
b2w is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther