Blog

A crypto wallet application to manage cryptocurrencies like Bitcoin, Ethereum, Ripple, and more. Secure.

Kinsta is a monitoring crawler that checks website availability, performance, and health.

Talkwalker delivers the consumer insights that help brands drive business impact. In a world full of conversations, the most successful global brands have switched to Talkwalker because we provide them with a powerful software platform to uncover, understand and derive the most valuable insights from internal and external data. Our listening and analytics platform enables more than 2,500 companies worldwide to protect their brands, measure their impact and gain the key consumer insights that drive purchase decisions.

Pulsetic is a monitoring crawler that checks website availability, performance, and health.

CCBot crawls the web for Common Crawl, a nonprofit that provides free web archives used to train many AI models including GPT and Claude.

Dataprovider.com collects web data for business intelligence, company research, and data enrichment services.

SE Ranking Bot is a monitoring crawler that checks website availability, performance, and health.

Google Digital Asset Links verifies app deep links and website associations for Android apps through assetlinks.json files.

OnCrawl is an enterprise SEO crawler that performs technical audits, log analysis, and data-driven SEO optimization.

Checkly is a high-programmability active monitoring solution. We support users in monitoring their websites and APIs. Puppeteer and Playwright (both supported) are browser automation tools that can be used for a variety of tasks. For testing, they really are about E2E/component testing, not unit testing.

AmazonAdBot is the web crawler used by Amazon's advertising services to visit and analyze advertiser landing pages to ensure compliance with advertising policies and optimize ad campaign performance.

Bing Preview generates page thumbnails and metadata for link previews in Bing search results, Outlook, Teams, and other Microsoft platforms.

Sogou web spider is the crawler for Sogou, a major Chinese search engine now part of Tencent, with over 10 billion Chinese pages indexed.

The SendGrid Event Webhook sends email event data to customer APIs as SendGrid processes it.

Uptime LLC is a monitoring crawler that checks website availability, performance, and health.

Grafana's Synthetic Monitoring is a monitoring crawler that checks website availability, performance, and health.

Yeti is Naver's web crawler that systematically discovers and indexes web content for South Korea's largest search engine. It crawls websites to build Naver's search index and support various platform services.

logicmonitor is a monitoring crawler that checks website availability, performance, and health.

FullStory is your digital experience analytics platform for on-the-fly funnels, pixel-perfect replay, custom events, heat maps, advanced search, Dev Tools, and more. FullStoryBot’s fetches and stores assets required to rebuild sites when viewing recorded sessions.

TTD-Content is a web crawler that classifies and analyzes webpage content for programmatic advertising purposes, helping advertisers determine brand-safe placement opportunities and content suitability for targeted advertising campaigns.