AI Agent-Readiness Bulk Auditor (llms.txt, Markdown, AI Bots)
Pricing
$50.00 / 1,000 domain auditeds
AI Agent-Readiness Bulk Auditor (llms.txt, Markdown, AI Bots)
Score any list of websites 0-100 on how well ChatGPT, Claude, Perplexity and Gemini agents can read them: robots.txt AI-bot blocks, firewall blocks, markdown negotiation, llms.txt, sitemap, JS-only pages, schema, hidden FAQs. Lead flag + pitch line per site.
Pricing
$50.00 / 1,000 domain auditeds
Rating
0.0
(0)
Developer
Growth Installations
Maintained by CommunityActor stats
0
Bookmarked
3
Total users
2
Monthly active users
13 hours ago
Last modified
Categories
Security
Limited permissionsShare
AI Agent-Readiness Bulk Auditor: llms.txt, Markdown, AI Bots, Schema
$0.05 per audited domain. Unreachable domains are free.
Paste a list of websites and get a 0-100 score for how well ChatGPT, Claude, Perplexity and Gemini agents can read each one. Every row includes the fixes, a lead flag and a pitch line you can use in cold email.
What does it do?
For every domain it runs about 8 light requests and checks:
| Check | Weight | What fails it |
|---|---|---|
| AI bots allowed in robots.txt | 20 | GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended and others disallowed |
| Firewall lets AI agents in | 10 | Homepage returns 403/429/challenge to an AI user agent but 200 to a browser |
| Markdown served | 15 | No markdown answer to Accept: text/markdown and no .md copy |
| /llms.txt | 10 | Missing or not markdown |
| Sitemap | 10 | No sitemap in robots.txt or at /sitemap.xml |
| Content in raw HTML | 15 | JS-only shell; agents that don't run JavaScript see an empty page |
| Schema | 10 | No Organization/LocalBusiness/Product/FAQ JSON-LD |
| FAQ readable in HTML | 5 | FAQ answers hidden behind JS accordions or missing FAQPage schema |
| Meta basics | 5 | Missing title, description or canonical |
Who is it for?
AI SEO / GEO agencies and AI-automation agencies that want a fast list of prospects with an obvious, fixable problem, and in-house SEOs auditing a portfolio of sites.
How much does it cost?
| Event | Price | What you get |
|---|---|---|
| Domain audited | $0.05 | One unique domain scored 0-100 with issues, fixes, lead flag and pitch line |
1,000 domains = $50. No third-party scrapers are used, so there are no extra upstream costs beyond normal Apify compute. Set a maximum charge per run in the Console to cap spend.
Input example
Output
One dataset row per domain: domain, score, grade, isLead, aiSearchBotsBlocked, aiBotsBlocked, aiUserAgentStatus, markdownServed, llmsTxt, sitemap, rawHtmlWords, jsShell, faqStatus, schemaTypes, emails, issueCount, issues, pitch, checks.
Key-value store: REPORT_MD, REPORT_HTML (white-label, print to PDF), REPORT_CSV.
Sample output (live run, Nov 3, 2026)
| domain | score | grade | isLead | top issue |
|---|---|---|---|---|
| stripe.com | 81 | B | false | No markdown served for Accept: text/markdown |
| brightlocal.com | 75 | B | false | robots.txt blocks Applebot-Extended, CCBot, meta-externalagent |
| gilmanheatingandcooling.com | 74 | B | false | No markdown version, no /llms.txt |
| ahrefs.com | 73 | B | false | No markdown version, no /llms.txt |
| anthropic.com | 63 | C | false | No markdown version, no /llms.txt, no schema |
| nectiv.com | 53 | D | true | No markdown, no /llms.txt, no sitemap, no schema |
Pitch line generated for nectiv.com: "Ran nectiv.com through an AI-agent readiness check: 53/100 (grade D). Biggest gaps: Serve a markdown version when requested with 'Accept: text/markdown'..."
FAQ
Does it crawl the whole site? No. It checks the homepage plus the robots.txt, llms.txt, sitemap and markdown endpoints, so it is fast and cheap for big lists.
Why is the score different with a proxy? Firewalls treat datacenter IPs differently. Leave the proxy off for the most accurate AI-agent firewall test.
Use it from n8n, Make, Claude or Cursor
- MCP: add
https://mcp.apify.com/?actors=growth-installations/agent-readiness-bulk-auditorto Claude, Cursor or any MCP client. - n8n or Make: add an HTTP Request step that POSTs your input JSON to the API endpoint below and returns the dataset items.
- API:
POST https://api.apify.com/v2/acts/growth-installations~agent-readiness-bulk-auditor/run-sync-get-dataset-itemswith headerAuthorization: Bearer <your Apify API token>
