1592+ Crawler profiliert
Crawler-Verzeichnis
Alle Web-Crawler und KI-Bots, die wir verfolgen. Was sie tun, wer sie betreibt und wie Sie Ihre Inhalte davor schützen.
infoobot — What It Is and How to Handle It
infoobot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
InfoNaviRobot — What It Is and How to Handle It
InfoNaviRobot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
INETDEX-BOT — What It Is and How to Handle It
INETDEX-BOT is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
IndeedJobBot — What It Is and How to Handle It
IndeedJobBot is Indeed's web scraper that crawls and indexes job postings across the internet to populate the Indeed job search platform with comprehensive employment opportunities.
AggregatorAggregator
IndeedBot — What It Is and How to Handle It
IndeedBot is Indeed's job search crawler that indexes job postings from company career pages for the Indeed job search engine.
AggregatorAggregator
imrbot — What It Is and How to Handle It
imrbot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
Impact.com Agent — What It Is and How to Handle It
Impact.com Agent is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
img2dataset — What It Is and How to Handle It
Image dataset crawler for ML training
AI CrawlerAI Training
imageSpider — What It Is and How to Handle It
imageSpider is a web crawler operated by ByteDance, the company behind TikTok, Douyin, and other content platforms. The bot collects images from websites across the internet, likely to support ByteDance's various AI products.
AI TrainingAI Training
ImagesiftBot — What It Is and How to Handle It
ImagesiftBot is a monitoring and analytics crawler that collects web data for business intelligence and competitive analysis.
MonitoringMonitoring
IFTTT — What It Is and How to Handle It
IFTTT is an RSS feed fetcher that monitors RSS and Atom feeds to trigger automated workflows and integrations. It crawls feeds to detect new content and executes user-configured automation triggers when new feed items are published.
Feed ReaderFeed Reader
ichiro — What It Is and How to Handle It
ichiro is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
ICBot — What It Is and How to Handle It
ICBot is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
IbouBot — What It Is and How to Handle It
IbouBot is a web crawler operated by Ibou that indexes publicly cited URLs to build a graph representation of the web for their search engine service. The bot crawls pages found on public websites to maintain and update their web database and metrics.
Search EngineSearch Engine
iaskspider — What It Is and How to Handle It
iaskspider is an AI search crawler for iAsk.ai that indexes content to power AI-generated answers with source citations.
AI CrawlerAI Assistant
iAskBot — What It Is and How to Handle It
iAskBot crawls websites for iAsk.ai, an AI-powered answer engine that provides instant answers by analyzing web content.
AI CrawlerAI Assistant
ias-va — What It Is and How to Handle It
ias-va is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
ias-ie — What It Is and How to Handle It
ias-ie is a web crawler operated by Integral Ad Science, a company that provides digital ad measurement and optimization services. The bot analyzes web page content to assess contextual relevance and brand safety, helping advertisers ensure their ads run in appropriate environments.
ResearchResearch
IABot — What It Is and How to Handle It
IABot (InternetArchiveBot) is a Wikipedia bot operated by the Internet Archive that combats link rot by finding dead links on Wikipedia articles and adding archived versions from the Wayback Machine to preserve reference accessibility.
ArchiverArchiver
ia_archiver-web.archive.org — What It Is and How to Handle It
ia_archiver-web.archive.org is a web archiver that captures and preserves website content for historical records.
ArchiverArchiver
ia_archiver — What It Is and How to Handle It
ia_archiver is a web archiver that captures and preserves website content for historical records.
ArchiverArchiver
hyscore.io — What It Is and How to Handle It
hyscore.io is a web crawler. Its specific purpose and operator are not publicly documented.
OtherOther
HyScore — What It Is and How to Handle It
HyScore is an uncategorized agent. If you think this is incorrect or can provide additional detail about its purpose, please let us know.
OtherOther
hypestat — What It Is and How to Handle It
hypestat analyzes website traffic, SEO metrics, and domain value estimates for HypeStat's website statistics platform.
SEO ToolSEO Tool
humanlinks — What It Is and How to Handle It
humanlinks is an SEO crawler that analyzes backlinks, website structure, and search optimization factors.
SEO ToolSEO Tool
Huckabot — What It Is and How to Handle It
Huckabot is Huckabuy's web crawler that indexes customer websites for dynamic rendering, page speed optimization, and SEO performance improvements through structured data enhancement.
Search EngineSearch Engine
HubSpot-FeedFetcher — What It Is and How to Handle It
HubSpot-FeedFetcher crawls and fetches images and content when posts are published from HubSpot to LinkedIn, ensuring proper content transfer.
Feed ReaderFeed Reader
HubSpot — What It Is and How to Handle It
HubSpot is a monitoring and analytics crawler that collects web data for business intelligence and competitive analysis.
MonitoringMonitoring
HTTrack 3.0 — What It Is and How to Handle It
HTTrack 3.0 is the legacy version of HTTrack website copier for downloading complete websites for offline access.
ArchiverArchiver
HTTrack — What It Is and How to Handle It
HTTrack is a free website copier that downloads websites for offline viewing, preserving site structure and links.
ArchiverArchiver