# Centinel Analytica > Centinel helps websites and applications evaluate automated traffic and apply access policies using request, browser, and session evidence. Centinel Analytica (Centinel) is a bot protection company based in Berlin. Centinel evaluates automated traffic and returns a policy decision for the site's integration to apply. Available evidence and enforcement depend on the deployment. Approved automation and abusive workflows need different policies. ## Product - [Centinel](https://www.centinelanalytica.com/): Bot protection for websites and applications. - [Bot management](https://www.centinelanalytica.com/bot-management): What bot management covers, the controls it should provide, and how to evaluate one. - [How it works](https://www.centinelanalytica.com/how-it-works): Follow request evidence, policy evaluation, and enforcement through the chosen integration. - [Integrations](https://www.centinelanalytica.com/integrations): Supported stacks and rollout preparation for an evaluation. - [Pricing](https://www.centinelanalytica.com/pricing): How to start, what drives the quote, and the procurement path. No public price list. - [Security](https://www.centinelanalytica.com/security): Certification and program status, what detection processes, and how to get the DPA and policies. - [Free AI crawler audit](https://www.centinelanalytica.com/free-audit): Get a manually reviewed report of the scraping tools that can access your content and where it is exposed. - [Book a demo](https://www.centinelanalytica.com/book-a-demo): Talk through your traffic with the Centinel team. ## Tools - [Tools](https://www.centinelanalytica.com/tools): The free tools, each with what it answers and where the pasted input goes. - [Crawler IP verifier](https://www.centinelanalytica.com/verify-crawler): Checks an address from your logs against the reverse DNS and published address ranges Google, OpenAI, Anthropic and Perplexity document themselves. - [User agent parser](https://www.centinelanalytica.com/user-agent-parser): Names the bot a user agent string claims to be, with its operator, type and robots.txt token. - [robots.txt checker](https://www.centinelanalytica.com/robots-txt-checker): Reads a site's robots.txt and reports what it says to each AI bot, and whether the site has an llms.txt. - [llms.txt generator](https://www.centinelanalytica.com/llms-txt-generator): Turns a sitemap or a list of page URLs into an llms.txt draft, grouped by section. ## Bot directory - [Bot directory](https://www.centinelanalytica.com/bots): Each common crawler's operator, user agent, robots.txt token and verification method. - [AI crawler list](https://www.centinelanalytica.com/ai-crawler-list): AI crawler user agents and robots.txt tokens. - [Google crawlers](https://www.centinelanalytica.com/bots/operator/google): The 11 Google bots in the directory, side by side. - [OpenAI crawlers](https://www.centinelanalytica.com/bots/operator/openai): The 3 OpenAI bots in the directory, side by side. - [Anthropic crawlers](https://www.centinelanalytica.com/bots/operator/anthropic): The 3 Anthropic bots in the directory, side by side. - [Meta crawlers](https://www.centinelanalytica.com/bots/operator/meta): The 4 Meta bots in the directory, side by side. - [360Spider](https://www.centinelanalytica.com/bots/360spider): 360 Search. Crawls pages for the web search of 360 Search (so.com). - [AdsBot-Google](https://www.centinelanalytica.com/bots/adsbot-google): Google. Checks the quality of ad landing pages for Google Ads. - [AhrefsBot](https://www.centinelanalytica.com/bots/ahrefsbot): Ahrefs. Builds the link database behind Ahrefs and the Yep search engine. - [AhrefsSiteAudit](https://www.centinelanalytica.com/bots/ahrefssiteaudit): Ahrefs. Crawls a site for an Ahrefs Site Audit project. - [Amazonbot](https://www.centinelanalytica.com/bots/amazonbot): Amazon. Crawls to improve Amazon products; may be used to train Amazon AI models. - [Applebot-Extended](https://www.centinelanalytica.com/bots/applebot-extended): Apple. A robots.txt token that opts content out of Apple model training. - [Applebot](https://www.centinelanalytica.com/bots/applebot): Apple. Crawls pages for Spotlight, Siri and Safari search features. - [Baiduspider](https://www.centinelanalytica.com/bots/baiduspider): Baidu. Crawls pages for Baidu search. - [Barkrowler](https://www.centinelanalytica.com/bots/barkrowler): Babbar. Collects links and page metadata for Babbar. - [Bingbot](https://www.centinelanalytica.com/bots/bingbot): Microsoft. Crawls pages for Bing search. - [Bytespider](https://www.centinelanalytica.com/bots/bytespider): Unconfirmed, attributed to ByteDance. A crawler with no operator documentation. - [CCBot](https://www.centinelanalytica.com/bots/ccbot): Common Crawl. Builds Common Crawl's open web archive, free for anyone to reuse. - [ChatGPT-User](https://www.centinelanalytica.com/bots/chatgpt-user): OpenAI. Visits a page when a ChatGPT or Custom GPT user's request needs it. - [Claude-SearchBot](https://www.centinelanalytica.com/bots/claude-searchbot): Anthropic. Indexes pages to improve Claude's search results. - [Claude-User](https://www.centinelanalytica.com/bots/claude-user): Anthropic. Fetches a page when a Claude user's question needs it. - [ClaudeBot](https://www.centinelanalytica.com/bots/claudebot): Anthropic. Collects web content that could contribute to Claude model training. - [DataForSeoBot](https://www.centinelanalytica.com/bots/dataforseobot): DataForSEO. Adds links to DataForSEO's backlink database. - [Diffbot](https://www.centinelanalytica.com/bots/diffbot): Diffbot. Crawls the web for the Diffbot Knowledge Graph and Diffbot's web search services. - [Discordbot](https://www.centinelanalytica.com/bots/discordbot): Discord. Visits a link shared in a Discord message to build its preview. - [DotBot](https://www.centinelanalytica.com/bots/dotbot): Moz. Gathers data for the Moz Link Index. - [DuckAssistBot](https://www.centinelanalytica.com/bots/duckassistbot): DuckDuckGo. Crawls in real time to support DuckDuckGo's AI-assisted answers. - [DuckDuckBot](https://www.centinelanalytica.com/bots/duckduckbot): DuckDuckGo. Crawls pages for DuckDuckGo search. - [facebookexternalhit](https://www.centinelanalytica.com/bots/facebookexternalhit): Meta. Builds link previews when content is shared on Meta's apps. - [Google-CloudVertexBot](https://www.centinelanalytica.com/bots/google-cloudvertexbot): Google. Crawls sites whose owners asked for it when building Vertex AI Agents. - [Google-Extended](https://www.centinelanalytica.com/bots/google-extended): Google. A robots.txt token that controls Gemini training and grounding use. - [Google-InspectionTool](https://www.centinelanalytica.com/bots/google-inspectiontool): Google. Fetches pages for Search Console URL inspection and the Rich Results Test. - [Googlebot-Image](https://www.centinelanalytica.com/bots/googlebot-image): Google. Crawls images for Google Images and image features in Search. - [Googlebot-News](https://www.centinelanalytica.com/bots/googlebot-news): Google. Controls whether your pages appear in Google News. - [Googlebot-Video](https://www.centinelanalytica.com/bots/googlebot-video): Google. Crawls video for the video features of Google Search. - [Googlebot](https://www.centinelanalytica.com/bots/googlebot): Google. Crawls pages for Google Search, Images, Video, News and Discover. - [GoogleOther](https://www.centinelanalytica.com/bots/googleother): Google. Fetches public pages for Google product teams, for example for research crawls. - [GPTBot](https://www.centinelanalytica.com/bots/gptbot): OpenAI. Crawls content that may be used to train OpenAI's foundation models. - [Kagibot](https://www.centinelanalytica.com/bots/kagibot): Kagi. Crawls pages for the Kagi search engine. - [Mediapartners-Google](https://www.centinelanalytica.com/bots/mediapartners-google): Google. Reads the pages of sites that show Google ads, so the ads match the content. - [Meta-ExternalAgent](https://www.centinelanalytica.com/bots/meta-externalagent): Meta. Crawls for training AI models or improving Meta products. - [Meta-ExternalFetcher](https://www.centinelanalytica.com/bots/meta-externalfetcher): Meta. Fetches individual links on behalf of Meta product users. - [Meta-WebIndexer](https://www.centinelanalytica.com/bots/meta-webindexer): Meta. Crawls pages to improve the search results of Meta AI. - [MistralAI-Index](https://www.centinelanalytica.com/bots/mistralai-index): Mistral AI. Indexes pages for Mistral search. - [MistralAI-User](https://www.centinelanalytica.com/bots/mistralai-user): Mistral AI. Fetches a page when a Mistral user's action needs it. - [MJ12bot](https://www.centinelanalytica.com/bots/mj12bot): Majestic. Maps links between websites for Majestic's link index. - [OAI-SearchBot](https://www.centinelanalytica.com/bots/oai-searchbot): OpenAI. Surfaces websites in ChatGPT's search features. - [Perplexity-User](https://www.centinelanalytica.com/bots/perplexity-user): Perplexity. Fetches a page when a Perplexity user's request needs it. - [PerplexityBot](https://www.centinelanalytica.com/bots/perplexitybot): Perplexity. Surfaces and links websites in Perplexity's search results. - [PetalBot](https://www.centinelanalytica.com/bots/petalbot): Huawei. Crawls pages for Petal Search and Huawei Assistant. - [Pinterestbot](https://www.centinelanalytica.com/bots/pinterestbot): Pinterest. Indexes pages behind Pins and keeps Pin details up to date. - [Rogerbot](https://www.centinelanalytica.com/bots/rogerbot): Moz. Crawls a site for the site audits of a Moz Pro Campaign. - [Screaming Frog SEO Spider](https://www.centinelanalytica.com/bots/screaming-frog-seo-spider): Screaming Frog. Audits a site's technical SEO from the machine that runs it. - [SemrushBot](https://www.centinelanalytica.com/bots/semrushbot): Semrush. Collects web data for Semrush's SEO tools. - [SeznamBot](https://www.centinelanalytica.com/bots/seznambot): Seznam.cz. Crawls pages for Seznam, a Czech search engine. - [Slackbot-LinkExpanding](https://www.centinelanalytica.com/bots/slackbot-linkexpanding): Slack. Fetches a page's meta tags when someone posts its link in Slack. - [Storebot-Google](https://www.centinelanalytica.com/bots/storebot-google): Google. Crawls product and store pages for Google Shopping. - [Twitterbot](https://www.centinelanalytica.com/bots/twitterbot): X. Fetches card markup when a link is shared on X. - [YandexBot](https://www.centinelanalytica.com/bots/yandexbot): Yandex. Yandex's main search indexing crawler. - [Yeti](https://www.centinelanalytica.com/bots/yeti): Naver. Crawls pages for Naver search in South Korea. - [YouBot](https://www.centinelanalytica.com/bots/youbot): You.com. Crawls and indexes pages for the You.com search engine. ## Use cases - [All use cases](https://www.centinelanalytica.com/use-cases): The automated-abuse problems Centinel is built for. - [Control what scrapers and AI crawlers can access](https://www.centinelanalytica.com/use-cases/scraping-and-ai-crawlers): Evaluate scraping and AI crawler protection: verify crawler identities, assess browser automation, and request a free audit of your site's exposed pages. - [Detect automation around high-risk login attempts](https://www.centinelanalytica.com/use-cases/account-takeover): Detect credential-stuffing automation with browser and session evidence while keeping login policy bounded and reviewable. - [Detect card-testing automation in web checkout](https://www.centinelanalytica.com/use-cases/checkout-and-carding-fraud): Detect card-testing automation around checkout and apply a route-specific response before repeated attempts reach payment processing. - [Detect automated sign-ups behind repeated promo claims](https://www.centinelanalytica.com/use-cases/promo-abuse-and-multi-accounting): Detect automated sign-ups used for promo abuse and apply a reviewable policy without claiming that every new account is fraudulent. - [Detect the scripts that trigger paid SMS verification codes](https://www.centinelanalytica.com/use-cases/sms-pumping): Detect SMS pumping automation on sign-up, login, and OTP routes, and challenge scripted code requests before your SMS provider sends and bills them. - [API abuse: apply request-level policy to direct API traffic](https://www.centinelanalytica.com/use-cases/api-abuse): Evaluate direct API traffic with request and identity evidence available to the server integration, alongside existing API security controls. - [Separate application-layer bot floods from volumetric DDoS](https://www.centinelanalytica.com/use-cases/ddos-and-traffic-floods): Evaluate application-layer bot traffic with automation evidence while retaining dedicated volumetric DDoS and availability protection. ## Industries - [All industries](https://www.centinelanalytica.com/industries): Listing platforms and the automation that targets them. - [Keep marketplace listings and messages free of automated abuse](https://www.centinelanalytica.com/industries/marketplaces): Bot protection for online marketplaces and classifieds: detect price scraping, fake seller accounts, and scripted message spam without friction for buyers. - [Protect real estate listings, agent contacts, and inquiries from bots](https://www.centinelanalytica.com/industries/real-estate-and-proptech): Bot protection for real estate portals and proptech marketplaces: detect real estate scraping, agent-contact harvesting, fake accounts, and inquiry spam. - [Protect product drops, checkout, and customer accounts from bots](https://www.centinelanalytica.com/industries/retail-and-ecommerce): Ecommerce bot protection: detect price scraping, inventory and sneaker bots, card testing, and credential stuffing without adding friction to checkout. - [Keep ticket on-sales for fans, not scalper bots](https://www.centinelanalytica.com/industries/ticketing-and-live-events): Bot protection for ticketing: detect scalper bots, seat-spinning holds, and bulk fan accounts at the on-sale without putting every fan through a CAPTCHA. - [Protect fares, seat inventory, and loyalty accounts from travel bots](https://www.centinelanalytica.com/industries/travel-and-airlines): Bot protection for airlines, hotels, and OTAs: detect fare scraping, seat-spinning holds, and loyalty account takeover, and keep metasearch partners. - [Decide which AI crawlers and scrapers can read your content](https://www.centinelanalytica.com/industries/media-and-publishing): Centinel is bot protection for publishers and media sites: verify AI crawlers, detect scrapers that ignore robots.txt, and keep search indexing. - [Detect scripted logins, sign-ups, and code requests at banks and fintechs](https://www.centinelanalytica.com/industries/financial-services): Bot protection for banks and fintechs: detect credential stuffing, scripted account opening, card testing, and OTP abuse, and keep approved aggregators. ## Guides - [All guides](https://www.centinelanalytica.com/blog): Every Centinel guide on bot detection, crawler policy, and automated abuse. - [Bot Mitigation for Publishers: Manage Unknown Crawlers at Edge](https://www.centinelanalytica.com/blog/bot-mitigation-for-publishers-manage-unknown-crawlers-at-edge): Publisher bot mitigation cannot treat unknown crawlers like known threats. Learn how to govern gray-zone traffic at the edge with clear rules and expiry dates. - [Form Bot Protection: Reduce Fake Leads Without Blocking Buyers](https://www.centinelanalytica.com/blog/form-bot-protection-reduce-fake-leads-without-blocking-buyers): Stop fake form submissions without breaking buyer conversions. Learn how server-side form bot protection blocks bots while preserving sales leads. - [Camoufox Paywall Audit: Test Publisher Access Controls](https://www.centinelanalytica.com/blog/camoufox-paywall-audit-test-publisher-access-controls): Conduct a Camoufox paywall audit on your media property. Test subscriber access controls, separate bot detection from authorization, and capture evidence. - [Bot False Positive Testing: Verify Visitors Before Blocking](https://www.centinelanalytica.com/blog/bot-false-positive-testing-verify-visitors-before-blocking): Run structured bot false positive testing before tightening edge WAF rules. Protect real readers, search bots, and subscribers with this publisher test matrix. - [Scraper Incident Response: Preserve Evidence Before Blocking](https://www.centinelanalytica.com/blog/scraper-incident-response-preserve-evidence-before-blocking): Execute a scraper incident response that preserves logs and forensic signals before blocking bots. Learn how to verify crawlers and stop bot IP rotation. - [Checkout Fraud Detection: Separate Bot Traffic from Payment Risk](https://www.centinelanalytica.com/blog/checkout-fraud-detection-separate-bot-traffic-from-payment-risk): Checkout fraud detection fails when you conflate bots with payment risk. Learn how to separate edge automation defense from transaction scoring. - [Price Scraping Protection: Test Controls on Product Pages](https://www.centinelanalytica.com/blog/price-scraping-protection-test-controls-on-product-pages): Test price scraping protection across product pages before rollout. Map endpoints, design layered limits, and audit false positives on real traffic. - [Residential Proxy Scraper Detection: How to Test Scraping Services](https://www.centinelanalytica.com/blog/residential-proxy-scraper-detection-how-to-test-scraping-services): Evaluate residential proxy scraper detection across scraping services and real traffic. Test detection stacks without blocking legitimate residential users. - [AI crawler policy for publishers: A newsroom access template](https://www.centinelanalytica.com/blog/ai-crawler-policy-for-publishers-a-newsroom-access-template): Build an AI crawler policy for publishers across editorial, legal, and tech. Classify bots, verify IPs, and stop scrapers without breaking search traffic. - [AI crawler costs: Build a business case from your own traffic logs](https://www.centinelanalytica.com/blog/ai-crawler-costs-build-a-business-case-from-your-own-traffic-logs): Calculate your true AI crawler costs from raw server logs and cloud invoices. A worksheet guide covering origin bandwidth, compute spend, and engineering time. - [AI browser agents: what a website can verify](https://www.centinelanalytica.com/blog/ai-browser-agents): Agentic browsers for site owners: ChatGPT's cloud browser signs its requests, while Comet and Claude in Chrome arrive as the user's own browser. - [What is card testing? Signs, costs and defenses](https://www.centinelanalytica.com/blog/what-is-card-testing): Card testing is scripted checking of stolen card details through a merchant's checkout or add-card flow. How it works, the signs, and what stops it. - [Cloudflare Bot Fight Mode vs. Bot Management](https://www.centinelanalytica.com/blog/cloudflare-bot-fight-mode-vs-bot-management): What Cloudflare's Bot Fight Mode, Super Bot Fight Mode, and Enterprise Bot Management each do, where each one stops, and when to move to the next level. - [Best bot management software: 10 tools compared](https://www.centinelanalytica.com/blog/best-bot-management-software): Ten bot management tools compared by deployment model, client-side signals, mobile and API coverage, and fit. Sourced from each vendor's own documentation. - [Pay per crawl explained: HTTP 402 and TollBit](https://www.centinelanalytica.com/blog/pay-per-crawl-explained): Pay per crawl charges AI crawlers per request. How Cloudflare, TollBit, RSL and x402 work, and why it depends on knowing which crawler is asking. - [Fake Googlebot: how to spot an impersonator](https://www.centinelanalytica.com/blog/fake-googlebot): Any client can send a Googlebot user agent. Verify the IP with reverse and forward DNS or Google's published ranges, and avoid the Google Cloud trap. - [What is ticket scalping? How scalper bots work](https://www.centinelanalytica.com/blog/what-is-ticket-scalping): Ticket scalping is buying limited tickets to resell at a markup, today mostly by bot. How scalper bots beat purchase limits, and what the BOTS Act prohibits. - [What is price scraping? How to spot it in logs](https://www.centinelanalytica.com/blog/what-is-price-scraping): Price scraping is the automated collection of prices from product, listing, or fare pages. See who runs it, how it shows up in logs, and which responses fit. - [What is Stagehand? The AI browser automation SDK](https://www.centinelanalytica.com/blog/what-is-stagehand): Stagehand is Browserbase's open-source SDK for driving Chromium with natural-language steps. How it works, where it runs, and what a target site can see. - [What is nodriver? Python CDP automation explained](https://www.centinelanalytica.com/blog/what-is-nodriver): nodriver is an async Python library that drives Chrome over the DevTools Protocol without Selenium or WebDriver. What it changes and what gives it away. - [What is Crawl4AI? The open-source AI crawler](https://www.centinelanalytica.com/blog/what-is-crawl4ai): Crawl4AI is an open-source Python crawler that turns web pages into Markdown for LLMs. What it can change, what your site sees, and how to assess it. - [Fake Googlebot and AI crawlers: 2026 data](https://www.centinelanalytica.com/blog/fake-crawler-report): On the typical news site, 1 in 7 requests claiming to be Googlebot failed verification, and 84% of GPTBot claims were fake. Data, method and CSV download. - [How to respond to a layer 7 bot attack](https://www.centinelanalytica.com/blog/respond-to-layer-7-bot-attacks): Contain an application-layer bot flood by finding the exhausted resource, narrowing traffic controls, and checking recovery. Includes an incident worksheet. - [How to detect cache-busting bots](https://www.centinelanalytica.com/blog/detect-cache-busting-bots): Investigate query-string variation, falling cache hits, and rising origin work. Review cache keys safely without merging meaningfully different responses. - [Bot traffic or traffic spike? How to tell](https://www.centinelanalytica.com/blog/bot-traffic-vs-traffic-spike): Compare traffic surges with request behavior, route cost, and real workflow completion. Avoid blocking a flash crowd or overlooking a bot-driven overload. - [GraphQL rate limiting for expensive queries](https://www.centinelanalytica.com/blog/prevent-graphql-abuse): Bound GraphQL query depth, breadth, pagination, and cost. Learn why HTTP request limits alone miss expensive operations. - [How to detect bot abuse on APIs](https://www.centinelanalytica.com/blog/detect-api-bot-abuse): Investigate API automation using credentials, routes, resource use, and outcomes. Distinguish approved clients from abusive requests. - [API rate limiting for bot protection](https://www.centinelanalytica.com/blog/api-rate-limiting-for-bot-protection): Choose API rate limits around client identity, route cost, bursts, and retries. Avoid one global rule that blocks users but misses expensive abuse. - [How to prevent automated promo-code abuse](https://www.centinelanalytica.com/blog/prevent-promo-code-abuse): Protect promotion eligibility and redemption from bots. Review repeated claims, concurrent requests, and retry handling without relying on IP alone. - [Fake accounts: how to detect automated signups](https://www.centinelanalytica.com/blog/detect-automated-account-creation): Fake accounts are registered in bulk, usually by scripts, for spam, fake reviews, or repeated sign-up credits. How to detect them from signup to first use. - [Card testing: how to prevent carding attacks](https://www.centinelanalytica.com/blog/how-to-prevent-carding-attacks): Card testing, or carding, uses scripts to check stolen card details against payment and card-setup flows. How to spot it and stop it before the provider call. - [How to prevent password-reset flooding and abuse](https://www.centinelanalytica.com/blog/prevent-password-reset-abuse): Protect recovery requests from automated flooding. Bound email delivery, retries, and token handling without letting attackers lock legitimate users out. - [How to prevent OTP request abuse and SMS pumping](https://www.centinelanalytica.com/blog/prevent-otp-request-abuse): Control automated one-time-code requests before delivery. Review resend budgets, destination patterns, and outcomes without blocking legitimate recovery. - [How to prevent account enumeration](https://www.centinelanalytica.com/blog/prevent-account-enumeration): Find account-existence leaks in login, signup, and password recovery. Review messages, status codes, and timing while keeping recovery usable. - [How to prevent account takeover](https://www.centinelanalytica.com/blog/how-to-prevent-account-takeover): Protect login, password recovery, and account changes with layered controls. Learn where bot protection helps and which identity checks remain essential. - [Credential stuffing: what it is, how to detect it](https://www.centinelanalytica.com/blog/detect-credential-stuffing): Credential stuffing is automated login testing with username and password pairs stolen from other sites. How it differs from brute force and how to detect it. - [How to investigate bot-detection false positives](https://www.centinelanalytica.com/blog/investigate-bot-detection-false-positives): Trace a legitimate user's block to the decision that caused it. Separate classification errors, policy mistakes, and integration failures. - [How to identify bot traffic in Google Analytics 4](https://www.centinelanalytica.com/blog/how-to-identify-bot-traffic-in-google-analytics): Spot bot traffic in GA4: rule out collection mistakes, build a slice of suspicious sessions, and confirm it against request logs before you block anything. - [CAPTCHA loop: why it repeats and how to fix it](https://www.centinelanalytica.com/blog/fix-bot-challenge-loops): A CAPTCHA challenge is a check a site runs before it lets a request through. Why visitors get stuck in repeat challenges, and how to find the failing step. - [CAPTCHA alternatives for your workflow](https://www.centinelanalytica.com/blog/captcha-alternatives): Compare non-interactive checks, proof of work, authentication, and workflow controls. Choose a CAPTCHA alternative by what it protects and how users recover. - [How to run a bot-protection proof of concept](https://www.centinelanalytica.com/blog/bot-protection-proof-of-concept): Build a buyer-owned bot-protection evaluation with clear workflows, labeled traffic, and enforcement checks. Includes a practical scorecard. - [How to protect site search from bot abuse](https://www.centinelanalytica.com/blog/protect-site-search-from-bots): Investigate automated search queries, deep pagination, and expensive filters. Set search-specific budgets while preserving useful search and autocomplete. - [Protect data-export endpoints from abuse](https://www.centinelanalytica.com/blog/protect-data-exports-from-abuse): Review export permissions, queue admission, result delivery, and signed downloads. Bound automated bulk extraction without disrupting approved data exports. - [How to prevent authenticated scraping](https://www.centinelanalytica.com/blog/prevent-authenticated-scraping): Control automated extraction by signed-in users. Review account permissions, resource budgets, exports, and approved integrations. - [How to prevent web scraping](https://www.centinelanalytica.com/blog/how-to-prevent-web-scraping): Choose scraping controls by route: access checks, rate limits, crawler rules, and bot detection. Use a control matrix to test failures and roll back safely. - [How to detect low-and-slow web scraping](https://www.centinelanalytica.com/blog/detect-low-and-slow-scraping): Find scraping that stays under short rate limits by comparing resource coverage, account activity, and extraction over a longer window than a rate chart. - [How to audit AI crawler traffic on your website](https://www.centinelanalytica.com/blog/audit-ai-crawler-traffic): Build a crawler inventory from request logs. Separate claimed names, verified operators, resource use, and policy outcomes before changing crawler access. - [What is an antidetect browser?](https://www.centinelanalytica.com/blog/what-is-an-antidetect-browser): An antidetect browser changes the browser characteristics a site can observe. Learn how profiles work, which inconsistencies matter, and how to respond. - [How to detect bots using residential proxies](https://www.centinelanalytica.com/blog/detect-bots-using-residential-proxies): Detect bots behind residential proxies using trusted source data, client consistency, and workflow evidence. Choose scoped controls without blocking whole ISPs. - [What is Zyte API? Scraping modes and detection](https://www.centinelanalytica.com/blog/what-is-zyte): Zyte API fetches raw HTTP, runs managed browsers and extracts structured data. What each mode looks like to a target site, and how to assess it. - [Ulixee Hero: the scraping browser and detection](https://www.centinelanalytica.com/blog/what-is-ulixee-hero): Ulixee Hero is a Chrome-based scraping browser that emulates an ordinary browser. What its emulation covers and how Centinel detected it in a 2026 test. - [Playwright Stealth: what it patches and misses](https://www.centinelanalytica.com/blog/what-is-playwright-stealth): playwright-stealth hides navigator.webdriver and other simple automation tells. What the Python package patches, what it cannot hide, how sites spot it. - [Kernel browser (kernel.sh): cloud Chromium](https://www.centinelanalytica.com/blog/what-is-kernel-sh): Kernel runs on-demand cloud Chromium for web agents. How it works, what its stealth mode changes, and how Centinel detected it in a September 2026 test. - [Camoufox in 2026: does it beat bot detection?](https://www.centinelanalytica.com/blog/what-is-camoufox): Camoufox is a Firefox fork that spoofs fingerprints in C++. What it changes, the leaks its docs admit, and how Centinel detected it in a 2026 test. - [What is Firecrawl? How it scrapes and is detected](https://www.centinelanalytica.com/blog/what-is-firecrawl): Firecrawl turns web pages into Markdown or structured data for AI agents. How it fetches, what your site sees, and how Centinel detected it in a 2026 test. - [What is Browserbase? Cloud browsers for AI agents](https://www.centinelanalytica.com/blog/what-is-browserbase): Browserbase rents remote Chromium sessions to AI agents over an API. How sessions work, what a target site sees, and how Centinel detected it in a 2026 test. - [Browser Use: how the AI browser agent gets caught](https://www.centinelanalytica.com/blog/what-is-browser-use): Browser Use lets a language model drive a real browser. How the agent works, what your logs see, and how Centinel detected it in a September 2026 test. - [Why WAF rules can miss browser-like crawlers](https://www.centinelanalytica.com/blog/why-your-waf-cant-see-them): Learn where request rules end, where session evidence can help, and how to test the bot controls enabled in your WAF without relying on category claims. - [What is agentic traffic? AI agents explained](https://www.centinelanalytica.com/blog/what-is-ai-agent-traffic): Agentic traffic comes from software that picks and performs steps toward a user's goal. How it differs from crawlers and fetchers, and how to measure it. - [How to control AI agent access](https://www.centinelanalytica.com/blog/how-to-control-ai-agent-access): Build a practical access policy for AI crawlers and agents with a resource matrix, explicit identity states, bounded responses, and tested failure behavior. - [Verify AI crawlers and agents: spot fake GPTBot](https://www.centinelanalytica.com/blog/how-to-verify-ai-agents): Check whether a request claiming to be GPTBot, ClaudeBot or another AI agent is real, using published IP ranges, reverse DNS or signed requests. - [Interstitial challenge pages for protection](https://www.centinelanalytica.com/blog/interstitial-page-content-protection): Decide when an interstitial challenge fits a protected route, which failure modes to handle, and how to measure the result before broad rollout. - [Automated browser detection: evidence and actions](https://www.centinelanalytica.com/blog/detect-browser-automation): Detect Playwright, Selenium and other automated browsers with request, browser and session evidence, then choose when to observe, challenge or block. - [Patched Chromium browsers explained](https://www.centinelanalytica.com/blog/patched-chromium-browsers): Learn what patched Chromium means, which automation signals a patch can change, and how to test browser-automation detection without relying on one property. - [TLS fingerprinting explained: JA3 vs JA4](https://www.centinelanalytica.com/blog/tls-fingerprinting-explained): A JA3 or JA4 fingerprint summarizes a TLS ClientHello. How each is built, how to read and capture one, and why a TLS fingerprint alone never proves a bot. - [GPTBot vs ClaudeBot vs Bytespider compared](https://www.centinelanalytica.com/blog/gptbot-vs-claudebot-vs-bytespider): Compare GPTBot, ClaudeBot and Bytespider: each operator's stated purpose, robots.txt guidance, published IP ranges, and how to check a claim. - [robots.txt for AI bots: tokens and examples](https://www.centinelanalytica.com/blog/robots-txt-ai-bots-guide): Copy-ready robots.txt rules for GPTBot, ClaudeBot, Google-Extended and other AI crawler tokens, how to test them, and why robots.txt cannot stop fakes. - [How to block AI crawlers and verify the block](https://www.centinelanalytica.com/blog/how-to-block-ai-crawlers): Block GPTBot, ClaudeBot and other AI crawlers: robots.txt rules by purpose, a verified request rule for copied names, and two curl checks to test it. - [What is web scraping?](https://www.centinelanalytica.com/blog/what-is-web-scraping): Understand how web scraping works, how it differs from crawling and browser automation, and how to build a resource-specific response. - [What is an AI crawler? Types and examples](https://www.centinelanalytica.com/blog/what-is-an-ai-crawler): AI crawlers collect web content for model development, AI search and retrieval. How they work, the main operators' crawlers, and what your logs can show. ## Topics - [AI crawlers and agents](https://www.centinelanalytica.com/blog/topic/ai-crawlers-and-agents): Guides to AI crawlers and agents: who operates them, how to verify the ones that identify themselves, and how to write and enforce an access policy. - [Scraping and automation tools](https://www.centinelanalytica.com/blog/topic/scraping-tools): What each scraping service and browser automation tool does, how its traffic looks to the site it visits, and which evidence separates it from a visitor. - [Web scraping protection](https://www.centinelanalytica.com/blog/topic/web-scraping): Guides to web scraping: how extraction works, where it shows in your traffic, and how to protect pages, search, exports, and signed-in data from it. - [Bot detection signals](https://www.centinelanalytica.com/blog/topic/detection-signals): How bot detection reads the connection, the browser, and the network: TLS fingerprints, patched browsers, residential proxies, and the limits of a WAF. - [Evaluating and running bot protection](https://www.centinelanalytica.com/blog/topic/evaluation-and-operations): Guides for choosing and operating bot protection: vendor shortlists, proof of concept, challenges, false positives, and bot traffic in your analytics. ## Compare - [All comparisons](https://www.centinelanalytica.com/compare): Centinel next to other bot-protection products, with the September 2026 per-tool benchmark. - [Centinel vs Akamai Bot Manager: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-akamai): Centinel and Akamai Bot Manager head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and Akamai 2. Fit, setup, and CSV. - [Centinel vs Cloudflare Bot Management: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-cloudflare): Centinel and Cloudflare Bot Management head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and Cloudflare 3. Setup and CSV. - [Centinel vs DataDome: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-datadome): Centinel and DataDome head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and DataDome 5. Fit, setup, migration, and CSV. - [Centinel vs Fastly Bot Management: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-fastly): Centinel and Fastly Bot Management head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and Fastly 0. Fit, setup, and CSV. - [Centinel vs HUMAN Security: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-human-security): Centinel and HUMAN Security (PerimeterX) head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and HUMAN 2. Setup and CSV. - [Centinel vs Kasada: 2026 Detection Test](https://www.centinelanalytica.com/compare/centinel-vs-kasada): Centinel and Kasada head to head: in our September 2026 test Centinel detected 8 of 8 scraping tools and Kasada 3. Fit, setup, migration, and CSV. - [Centinel vs. CDN and WAF bot protection](https://www.centinelanalytica.com/compare/centinel-vs-cdn-bot-protection): A practical method for deciding whether Centinel adds useful automation evidence to the CDN, WAF, and bot controls you already operate. - [Cloudflare Bot Management Alternatives, Tested](https://www.centinelanalytica.com/alternatives/cloudflare): Cloudflare Bot Management alternatives compared on one September 2026 test of eight scraping tools, and how to add one without leaving Cloudflare. - [DataDome Alternatives: 2026 Detection Test](https://www.centinelanalytica.com/alternatives/datadome): DataDome alternatives compared on one September 2026 test of eight scraping tools, with deployment, buying model, and EU data notes for each option. - [HUMAN Security Alternatives: 2026 Detection Test](https://www.centinelanalytica.com/alternatives/human-security): HUMAN Security (PerimeterX) alternatives compared on one September 2026 test of eight scraping tools, with deployment and buying notes for each option. ## Company - [About](https://www.centinelanalytica.com/about): The team and mission behind Centinel. - [Frederick Jahn](https://www.centinelanalytica.com/about#frederick-jahn): Managing director and author of Centinel's technical guides. - [Careers](https://www.centinelanalytica.com/careers): Help build bot protection for site owners. - [Contact](https://www.centinelanalytica.com/contact): Talk with the Centinel team. ## Deutsch - [https://www.centinelanalytica.com/de](https://www.centinelanalytica.com/de): German edition of https://www.centinelanalytica.com/ - [https://www.centinelanalytica.com/de/bot-erkennung](https://www.centinelanalytica.com/de/bot-erkennung): German edition of https://www.centinelanalytica.com/bot-management - [https://www.centinelanalytica.com/de/branchen/medien-und-verlage](https://www.centinelanalytica.com/de/branchen/medien-und-verlage): German edition of https://www.centinelanalytica.com/industries/media-and-publishing - [https://www.centinelanalytica.com/de/ki-crawler](https://www.centinelanalytica.com/de/ki-crawler): German edition of https://www.centinelanalytica.com/blog/what-is-an-ai-crawler - [https://www.centinelanalytica.com/de/studie/gefaelschte-crawler](https://www.centinelanalytica.com/de/studie/gefaelschte-crawler): German edition of https://www.centinelanalytica.com/blog/fake-crawler-report - [https://www.centinelanalytica.com/de/crawler-pruefen](https://www.centinelanalytica.com/de/crawler-pruefen): German edition of https://www.centinelanalytica.com/verify-crawler - [https://www.centinelanalytica.com/de/kostenloser-audit](https://www.centinelanalytica.com/de/kostenloser-audit): German edition of https://www.centinelanalytica.com/free-audit - [https://www.centinelanalytica.com/de/demo](https://www.centinelanalytica.com/de/demo): German edition of https://www.centinelanalytica.com/book-a-demo - [https://www.centinelanalytica.com/de/datenschutz-dsgvo](https://www.centinelanalytica.com/de/datenschutz-dsgvo) ## Markdown Each bot, tool, guide and comparison page above also answers as Markdown: add `.md` to its URL, or send `Accept: text/markdown`. - [Full text](https://www.centinelanalytica.com/llms-full.txt): Those pages in one Markdown file. ## Optional - [Full sitemap](https://www.centinelanalytica.com/sitemap.xml)