Check if ChatGPT, Claude, Perplexity & Google AI Overviews can find and cite your site: robots.txt matrix for 20 AI bots, llms.txt, CDN blocking signals, JS-only content, structured data and trust signals. Scores 0-100 with copy-paste fixes.
URL privacy: the dataset, reports, OUTPUT, monitoring snapshots, alerts and the run log show URLs without query strings, fragments or credentials (?[redacted]); each row has a urlHash (first 16 hex characters of the SHA-256 of the checked URL) to match rows to your input. Same-page links (#anchor, empty or same-query links) are left out of the llms.txt draft and link lists, so the query string of the URL you entered is not repeated there.
Every start URL is checked before anything is loaded (IP literals in every encoding, internal names, DNS answer); a private, loopback, link-local, cloud-metadata or internal target gets an error row "refused: private-network protection; not charged" and is never contacted or charged.
All plain HTTP fetches (pages, robots.txt, sitemaps, llms.txt, access-log download, IP-range lists) use a DNS-pinned transport that re-checks every redirect hop; the same applies to Chromium (rendering, screenshots, PDF logos), which now runs through a local checking proxy (shared SiteVerdict guard, src/ssrf-guard.ts + src/netguard.ts).
webhookUrl, slackWebhookUrl, reportLogoUrl and accessLogUrl must resolve to public hosts; otherwise they are ignored with a warning. Webhooks never follow redirects.
OUTPUT gains privateNetworkProtection (refused targets, browser requests blocked). No change to scoring or pricing.
Unreleased (prepared 2026-10-16, NOT pushed) — URL privacy and reporting fixes
Dataset url / inputUrl, per-page url, OUTPUT.results[].url, reports, monitoring snapshots, alerts and log lines no longer contain the query string, fragment or credentials of the URL you entered: they become ?[redacted] / #[redacted], paths are capped at 120 characters. New urlHash field (first 16 hex of SHA-256 of the full checked URL) as a join key. Monitoring baselines are still keyed by the full URL, so existing baselines keep matching.
Final URLs, request evidence and error texts are shown without query strings; the suggested og:url and canonical snippets use the page address without its query string.
OUTPUT.privateNetworkProtection.browserRequestsBlocked is now filled in (it was read after the browser had closed and was always empty).
No change to scoring, inputs or pricing.
Memory default (15 Oct 2026)
15 Oct 2026 (build after this entry): default run memory is now 2048 MB via defaultMemoryMbytes (was 4096 MB (no key set); peak 260 MB). Measured on the example input: same output, see the public-15 remediation log. You can still choose more memory per run. No change to inputs, outputs or pricing.
2.0.0 — 2026-10-11
Flagship upgrade. Benchmarked on 25 public sites; see BENCHMARK.md.
Removed (compliance)
The opt-in CDN probe that sent other vendors' AI user-agent strings is gone. We never impersonate crawlers.
AI crawlers
35 agents, each re-verified against vendor documentation on 11 Oct 2026. New: Google-Agent, Google-GeminiNotebook, Meta-WebIndexer, MistralAI-Index/User/Training, Amzn-User, YouBot, OAI-AdsBot, GoogleOther, Google-CloudVertexBot and facebookexternalhit.
Every agent now carries a role (search / user / training / other), a status (official / unofficial / legacy) and per-row confidence.
CDN / WAF AI blocking (new honest model)
Verdict levels: confirmed / likely / possible / unlikely, built from evidence:
challenges, 401/403/429 and 402 served to our honest client
Cloudflare AI Crawl Control defaults
Cloudflare managed robots.txt
Access-log verification (new): combined-log or JSON-lines files (e.g. Cloudflare Logpush). Each request's source IP is checked against the vendors' official IP-range lists and the result shows the status codes real AI bots received.
Bot-challenge detection now also catches Akamai bot-failover pages served with HTTP 200.
AI usage preferences (new)
Reads Cloudflare Content-Signal (per group), IETF aipref Content-Usage lines and W3C TDMRep (header, meta tag and /.well-known/tdmrep.json).
Raw vs JavaScript-rendered (rewritten)
Reports the share of rendered text present in the raw HTML, plus structured data and headings that exist only after JavaScript.
Flags title, meta, canonical, noindex and nosnippet changes made by JavaScript.
Optional screenshot evidence.
Entity and structured data
Checks the @id graph, WebSite→publisher links, dangling references, knowledge-graph sameAs and NAP phone consistency.
More required-property checks: Event, Recipe, JobPosting, Offer, Person.
A bare {"@id"} reference node is no longer reported as missing @type.
First release: robots.txt matrix for 20 AI agents, passive CDN detection, opt-in UA probe, raw/rendered word ratio, structure and trust checks, llms.txt validation, 0–100 score, top fixes, HTML report.