Store/Crawler Surface Preflight

LLMS.txt Auditor

junipr/llms-txt-auditor

Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

Find public llms.txt, llms-full.txt, and AI-readable documentation links for company domains.

AI Visibility Site Audit — robots.txt & llms.txt checks

43sunsets/ai-visibility-site-audit

Audit how visible your own website is to AI crawlers and assistants: robots.txt rules for 13 AI bots, llms.txt, structured data, meta hygiene and sitemap — with a scored report and a ready-to-edit llms.txt draft.

Check robots.txt, llms.txt, sitemap hints, and AI crawler blocking rules for websites and domains.

Audit domains for llms.txt, robots.txt AI bot rules, sitemap hints, and public AI crawler readiness signals.

Analyze public robots.txt and llms.txt files for AI crawler allow/block policy evidence, LLM guidance files, stable hashes, and useful-result pricing.

Scan robots.txt files in bulk to extract sitemap URLs and verify crawler directives for technical SEO compliance.

Discover robots.txt rules, sitemap URLs, crawl delays, and basic crawlability signals for domains or website URLs.

Check websites for llms.txt and llms-full.txt, extract sections and links for AI readiness and GEO audits.

Discover every URL a site exposes via its public sitemap chain. Reads robots.txt, follows Sitemap declarations, recursively descends sitemap-index files, extracts URLs with lastmod, changefreq, priority.