AI Visibility Site Audit — robots.txt & llms.txt checks
Pricing
from $250.00 / 1,000 report generateds
AI Visibility Site Audit — robots.txt & llms.txt checks
Audit how visible your own website is to AI crawlers and assistants: robots.txt rules for 13 AI bots, llms.txt, structured data, meta hygiene and sitemap — with a scored report and a ready-to-edit llms.txt draft.
Pricing
from $250.00 / 1,000 report generateds
Rating
0.0
(0)
Developer
Ken Yao
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
2 days ago
Last modified
Categories
Share
AI Visibility Site Audit
Find out — in one run — whether AI crawlers and assistants can actually see your website, and what they see.
AI assistants are becoming a real discovery channel. But their crawlers read your site differently than Google does, and most sites have never checked what those bots are allowed to access. This Actor audits your own site and returns a scored report (0–100) plus a ready-to-edit llms.txt draft.
What it checks
- AI crawler access — your
robots.txtrules evaluated for 13 AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-Web, anthropic-ai, Google-Extended, PerplexityBot, CCBot, Bytespider, Amazonbot, Applebot-Extended, meta-externalagent). Blocked, partial or allowed — per bot, with the exact rule that applies. - llms.txt / llms-full.txt — present or not, plus a generated draft built from your actual pages.
- Structured data — JSON-LD presence and types per page (unparseable blocks are flagged too).
- Titles & meta descriptions — presence and length hygiene on every crawled page.
- Canonical & indexability — canonical tags and accidental
noindex. - sitemap.xml — present or not.
- Thin / JS-rendered content — pages whose raw HTML contains almost no text, which is what many bots actually see.
What you get
REPORT(Markdown, in the run's key-value store): the full scored audit with per-bot table, per-page findings, and your llms.txt draft.- Dataset: one row per crawled page with all checks, ready to sort and filter.
OUTPUT: the summary object (score, grade, category breakdown) for automation.
Input
| Field | Description |
|---|---|
startUrl | Your website. The crawl stays on this hostname only. |
maxPages | Page limit (default 25, max 200). Each crawled page is one billed event. |
Honest notes
- llms.txt is an emerging convention. Google has publicly stated it does not affect ranking, and adoption by AI tools is uneven. We report it as low-cost hygiene, not a magic growth switch.
- Static analysis. Pages are fetched as raw HTML without JavaScript rendering — which is exactly how many crawlers see them. Pages that depend on JS are flagged rather than rendered.
- Blocking AI bots can be the right choice. The audit makes your current state visible so it is a decision, not an accident.
- The Actor audits only the site you provide, stays on the same hostname, and respects robots.txt during its own crawl.
Pricing
Pay per event: a small start fee, a per-page fee for each crawled page, and a report fee. Control cost with maxPages — a 25-page audit costs well under a dollar.
