AI Crawler Access Audit
Pricing
from $3.00 / 1,000 site audits
Go to Apify Store
AI Crawler Access Audit
See which AI and search crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot and 150+ more) each site allows or blocks in robots.txt, and whether llms.txt is published. One row per URL, at most two small requests per site, no page content.