Oct 11, 2026
AI browser agents: what a website can verify
Agentic browsers for site owners: ChatGPT's cloud browser signs its requests, while Comet and Claude in Chrome arrive as the user's own browser.
Frederick Jahn
Understand the threats
Practical guides on AI crawlers, web scraping, and content protection.
AI crawlers and agents
Sep 30, 2026
Pay per crawl explained: HTTP 402 and TollBit
Pay per crawl charges AI crawlers per request. How Cloudflare, TollBit, RSL and x402 work, and why it depends on knowing which crawler is asking.
Frederick Jahn
Sep 30, 2026
Fake Googlebot: how to spot an impersonator
Any client can send a Googlebot user agent. Verify the IP with reverse and forward DNS or Google's published ranges, and avoid the Google Cloud trap.
Frederick Jahn
Sep 22, 2026
Fake Googlebot and AI crawlers: 2026 data
On the typical news site, 1 in 7 requests claiming to be Googlebot failed verification, and 84% of GPTBot claims were fake. Data, method and CSV download.
Frederick Jahn
Sep 5, 2026
How to audit AI crawler traffic on your website
Build a crawler inventory from request logs. Separate claimed names, verified operators, resource use, and policy outcomes before changing crawler access.
Frederick Jahn
Apr 20, 2026
What is agentic traffic? AI agents explained
Agentic traffic comes from software that picks and performs steps toward a user's goal. How it differs from crawlers and fetchers, and how to measure it.
Frederick Jahn
Apr 20, 2026
How to control AI agent access
Build a practical access policy for AI crawlers and agents with a resource matrix, explicit identity states, bounded responses, and tested failure behavior.
Frederick Jahn
Apr 20, 2026
Verify AI crawlers and agents: spot fake GPTBot
Check whether a request claiming to be GPTBot, ClaudeBot or another AI agent is real, using published IP ranges, reverse DNS or signed requests.
Frederick Jahn
Mar 18, 2026
GPTBot vs ClaudeBot vs Bytespider compared
Compare GPTBot, ClaudeBot and Bytespider: each operator's stated purpose, robots.txt guidance, published IP ranges, and how to check a claim.
Frederick Jahn
Mar 14, 2026
robots.txt for AI bots: tokens and examples
Copy-ready robots.txt rules for GPTBot, ClaudeBot, Google-Extended and other AI crawler tokens, how to test them, and why robots.txt cannot stop fakes.
Frederick Jahn
Mar 12, 2026
How to block AI crawlers and verify the block
Block GPTBot, ClaudeBot and other AI crawlers: robots.txt rules by purpose, a verified request rule for copied names, and two curl checks to test it.
Frederick Jahn
Mar 10, 2026
What is an AI crawler? Types and examples
AI crawlers collect web content for model development, AI search and retrieval. How they work, the main operators' crawlers, and what your logs can show.
Frederick Jahn
Scraping and automation tools
Sep 24, 2026
What is Stagehand? The AI browser automation SDK
Stagehand is Browserbase's open-source SDK for driving Chromium with natural-language steps. How it works, where it runs, and what a target site can see.
Frederick Jahn
Sep 24, 2026
What is nodriver? Python CDP automation explained
nodriver is an async Python library that drives Chrome over the DevTools Protocol without Selenium or WebDriver. What it changes and what gives it away.
Frederick Jahn
Sep 24, 2026
What is Crawl4AI? The open-source AI crawler
Crawl4AI is an open-source Python crawler that turns web pages into Markdown for LLMs. What it can change, what your site sees, and how to assess it.
Frederick Jahn
Sep 3, 2026
What is Zyte API? Scraping modes and detection
Zyte API fetches raw HTTP, runs managed browsers and extracts structured data. What each mode looks like to a target site, and how to assess it.
Frederick Jahn
Sep 3, 2026
Ulixee Hero: the scraping browser and detection
Ulixee Hero is a Chrome-based scraping browser that emulates an ordinary browser. What its emulation covers and how Centinel detected it in a 2026 test.
Frederick Jahn
Sep 3, 2026
Playwright Stealth: what it patches and misses
playwright-stealth hides navigator.webdriver and other simple automation tells. What the Python package patches, what it cannot hide, how sites spot it.
Frederick Jahn
Sep 3, 2026
Kernel browser (kernel.sh): cloud Chromium
Kernel runs on-demand cloud Chromium for web agents. How it works, what its stealth mode changes, and how Centinel detected it in a September 2026 test.
Frederick Jahn
Sep 3, 2026
Camoufox in 2026: does it beat bot detection?
Camoufox is a Firefox fork that spoofs fingerprints in C++. What it changes, the leaks its docs admit, and how Centinel detected it in a 2026 test.
Frederick Jahn
Sep 2, 2026
What is Firecrawl? How it scrapes and is detected
Firecrawl turns web pages into Markdown or structured data for AI agents. How it fetches, what your site sees, and how Centinel detected it in a 2026 test.
Frederick Jahn
Sep 2, 2026
What is Browserbase? Cloud browsers for AI agents
Browserbase rents remote Chromium sessions to AI agents over an API. How sessions work, what a target site sees, and how Centinel detected it in a 2026 test.
Frederick Jahn
Sep 2, 2026
Browser Use: how the AI browser agent gets caught
Browser Use lets a language model drive a real browser. How the agent works, what your logs see, and how Centinel detected it in a September 2026 test.
Frederick Jahn
Web scraping protection
Sep 24, 2026
What is price scraping? How to spot it in logs
Price scraping is the automated collection of prices from product, listing, or fare pages. See who runs it, how it shows up in logs, and which responses fit.
Frederick Jahn
Sep 5, 2026
How to protect site search from bot abuse
Investigate automated search queries, deep pagination, and expensive filters. Set search-specific budgets while preserving useful search and autocomplete.
Frederick Jahn
Sep 5, 2026
Protect data-export endpoints from abuse
Review export permissions, queue admission, result delivery, and signed downloads. Bound automated bulk extraction without disrupting approved data exports.
Frederick Jahn
Sep 5, 2026
How to prevent authenticated scraping
Control automated extraction by signed-in users. Review account permissions, resource budgets, exports, and approved integrations.
Frederick Jahn
Sep 5, 2026
How to prevent web scraping
Choose scraping controls by route: access checks, rate limits, crawler rules, and bot detection. Use a control matrix to test failures and roll back safely.
Frederick Jahn
Sep 5, 2026
How to detect low-and-slow web scraping
Find scraping that stays under short rate limits by comparing resource coverage, account activity, and extraction over a longer window than a rate chart.
Frederick Jahn
Apr 1, 2026
Interstitial challenge pages for protection
Decide when an interstitial challenge fits a protected route, which failure modes to handle, and how to measure the result before broad rollout.
Frederick Jahn
Mar 10, 2026
What is web scraping?
Understand how web scraping works, how it differs from crawling and browser automation, and how to build a resource-specific response.
Frederick Jahn
Bot detection signals
Sep 4, 2026
How to detect bots using residential proxies
Detect bots behind residential proxies using trusted source data, client consistency, and workflow evidence. Choose scoped controls without blocking whole ISPs.
Frederick Jahn
Jul 22, 2026
Why WAF rules can miss browser-like crawlers
Learn where request rules end, where session evidence can help, and how to test the bot controls enabled in your WAF without relying on category claims.
Frederick Jahn
Mar 28, 2026
Automated browser detection: evidence and actions
Detect Playwright, Selenium and other automated browsers with request, browser and session evidence, then choose when to observe, challenge or block.
Frederick Jahn
Mar 25, 2026
Patched Chromium browsers explained
Learn what patched Chromium means, which automation signals a patch can change, and how to test browser-automation detection without relying on one property.
Frederick Jahn
Mar 22, 2026
TLS fingerprinting explained: JA3 vs JA4
A JA3 or JA4 fingerprint summarizes a TLS ClientHello. How each is built, how to read and capture one, and why a TLS fingerprint alone never proves a bot.
Frederick Jahn
Evaluating and running bot protection
Sep 30, 2026
Cloudflare Bot Fight Mode vs. Bot Management
What Cloudflare's Bot Fight Mode, Super Bot Fight Mode, and Enterprise Bot Management each do, where each one stops, and when to move to the next level.
Frederick Jahn
Sep 30, 2026
Best bot management software: 10 tools compared
Ten bot management tools compared by deployment model, client-side signals, mobile and API coverage, and fit. Sourced from each vendor's own documentation.
Frederick Jahn
Sep 5, 2026
How to investigate bot-detection false positives
Trace a legitimate user's block to the decision that caused it. Separate classification errors, policy mistakes, and integration failures.
Frederick Jahn
Sep 5, 2026
How to identify bot traffic in Google Analytics 4
Spot bot traffic in GA4: rule out collection mistakes, build a slice of suspicious sessions, and confirm it against request logs before you block anything.
Frederick Jahn
Sep 5, 2026
CAPTCHA loop: why it repeats and how to fix it
A CAPTCHA challenge is a check a site runs before it lets a request through. Why visitors get stuck in repeat challenges, and how to find the failing step.
Frederick Jahn
Sep 5, 2026
CAPTCHA alternatives for your workflow
Compare non-interactive checks, proof of work, authentication, and workflow controls. Choose a CAPTCHA alternative by what it protects and how users recover.
Frederick Jahn
Sep 5, 2026
How to run a bot-protection proof of concept
Build a buyer-owned bot-protection evaluation with clear workflows, labeled traffic, and enforcement checks. Includes a practical scorecard.
Frederick Jahn
Account takeover
Sep 5, 2026
How to prevent password-reset flooding and abuse
Protect recovery requests from automated flooding. Bound email delivery, retries, and token handling without letting attackers lock legitimate users out.
Frederick Jahn
Sep 5, 2026
How to prevent OTP request abuse and SMS pumping
Control automated one-time-code requests before delivery. Review resend budgets, destination patterns, and outcomes without blocking legitimate recovery.
Frederick Jahn
Sep 5, 2026
How to prevent account enumeration
Find account-existence leaks in login, signup, and password recovery. Review messages, status codes, and timing while keeping recovery usable.
Frederick Jahn
Sep 5, 2026
How to prevent account takeover
Protect login, password recovery, and account changes with layered controls. Learn where bot protection helps and which identity checks remain essential.
Frederick Jahn
Sep 5, 2026
Credential stuffing: what it is, how to detect it
Credential stuffing is automated login testing with username and password pairs stolen from other sites. How it differs from brute force and how to detect it.
Frederick Jahn
Sep 4, 2026
What is an antidetect browser?
An antidetect browser changes the browser characteristics a site can observe. Learn how profiles work, which inconsistencies matter, and how to respond.
Frederick Jahn
Checkout and carding fraud
Sep 30, 2026
What is card testing? Signs, costs and defenses
Card testing is scripted checking of stolen card details through a merchant's checkout or add-card flow. How it works, the signs, and what stops it.
Frederick Jahn
Sep 5, 2026
Card testing: how to prevent carding attacks
Card testing, or carding, uses scripts to check stolen card details against payment and card-setup flows. How to spot it and stop it before the provider call.
Frederick Jahn
Promo abuse and multi-accounting
Sep 5, 2026
How to prevent automated promo-code abuse
Protect promotion eligibility and redemption from bots. Review repeated claims, concurrent requests, and retry handling without relying on IP alone.
Frederick Jahn
Sep 5, 2026
Fake accounts: how to detect automated signups
Fake accounts are registered in bulk, usually by scripts, for spam, fake reviews, or repeated sign-up credits. How to detect them from signup to first use.
Frederick Jahn
API abuse
Sep 5, 2026
GraphQL rate limiting for expensive queries
Bound GraphQL query depth, breadth, pagination, and cost. Learn why HTTP request limits alone miss expensive operations.
Frederick Jahn
Sep 5, 2026
How to detect bot abuse on APIs
Investigate API automation using credentials, routes, resource use, and outcomes. Distinguish approved clients from abusive requests.
Frederick Jahn
Sep 5, 2026
API rate limiting for bot protection
Choose API rate limits around client identity, route cost, bursts, and retries. Avoid one global rule that blocks users but misses expensive abuse.
Frederick Jahn
DDoS and traffic floods
Sep 5, 2026
How to respond to a layer 7 bot attack
Contain an application-layer bot flood by finding the exhausted resource, narrowing traffic controls, and checking recovery. Includes an incident worksheet.
Frederick Jahn
Sep 5, 2026
How to detect cache-busting bots
Investigate query-string variation, falling cache hits, and rising origin work. Review cache keys safely without merging meaningfully different responses.
Frederick Jahn
Sep 5, 2026
Bot traffic or traffic spike? How to tell
Compare traffic surges with request behavior, route cost, and real workflow completion. Avoid blocking a flash crowd or overlooking a bot-driven overload.
Frederick Jahn
Ticketing and live events
Other research
Oct 15, 2026
Bot False Positive Testing: Verify Visitors Before Blocking
Run structured bot false positive testing before tightening edge WAF rules. Protect real readers, search bots, and subscribers with this publisher test matrix.
Centinel Analytica
Oct 15, 2026
Scraper Incident Response: Preserve Evidence Before Blocking
Execute a scraper incident response that preserves logs and forensic signals before blocking bots. Learn how to verify crawlers and stop bot IP rotation.
Centinel Analytica
Oct 13, 2026
Checkout Fraud Detection: Separate Bot Traffic from Payment Risk
Checkout fraud detection fails when you conflate bots with payment risk. Learn how to separate edge automation defense from transaction scoring.
Centinel Analytica
Oct 13, 2026
Price Scraping Protection: Test Controls on Product Pages
Test price scraping protection across product pages before rollout. Map endpoints, design layered limits, and audit false positives on real traffic.
Centinel Analytica
Oct 11, 2026
Residential Proxy Scraper Detection: How to Test Scraping Services
Evaluate residential proxy scraper detection across scraping services and real traffic. Test detection stacks without blocking legitimate residential users.
Centinel Analytica
Oct 11, 2026
AI crawler policy for publishers: A newsroom access template
Build an AI crawler policy for publishers across editorial, legal, and tech. Classify bots, verify IPs, and stop scrapers without breaking search traffic.
Centinel Analytica
Oct 11, 2026
AI crawler costs: Build a business case from your own traffic logs
Calculate your true AI crawler costs from raw server logs and cloud invoices. A worksheet guide covering origin bandwidth, compute spend, and engineering time.
Centinel Analytica