Control what scrapers and AI crawlers can access
Scrapers and AI crawlers copy your prices, product catalogue, and articles page by page, often through a real browser on residential IP addresses. Competitors undercut you with your own prices, and AI services answer with your content without sending the reader to your site.
Request a site auditA crawler name is a claim, not proof
Copied pages, repeated catalogue requests, and unexpected origin load can indicate scraping. A crawler name does not show who sent the request. Check a declared client against its operator's published data, and look for automation evidence in a disguised scraper. Search crawlers, monitoring tools, and partners may still need access. Decide which resources each client can reach.
Read the analysisWhat the traffic looks like
A browser, not a script
Automation frameworks can drive a full browser, execute page scripts, and produce requests that resemble an ordinary visit.
A different IP on every request
Residential proxy networks can rotate requests across consumer connections, which makes a static address rule incomplete.
A crawler name anyone can claim
Anyone can copy a user-agent string. Check the request against the operator's published verification data, such as IP ranges or DNS records.
What runs against this traffic
Verify declared identities
Centinel can compare a declared crawler identity with verification data published by its operator.
A matching identity can receive its intended policy. An unverified claim receives no automatic trust.
Read more: Verify declared identitiesEvaluate browser evidence
In supported browser deployments, Centinel combines request and browser evidence instead of relying on one automation signal.
Teams can challenge or restrict suspicious automation without putting every visitor through the same check.
Read more: Evaluate browser evidenceA policy for each traffic class
Supported integrations apply the configured response for the client's identity, behaviour, and requested resource.
Approved crawlers can keep the access they need without extending that trust to disguised scrapers. Confirm the enforcement point and available responses for your stack before changing policy.
Read more: A policy for each traffic class
See what reaches your site
Start with a free scraping audit. A person reviews the scraping test results, then emails you the exposed pages and fixes to consider. The audit covers scraping exposure only, not every abuse pattern or all production traffic. Discuss broader coverage in a demo.