Website AI Content & Data Leakage Scanner

Find public llms.txt, llms-full.txt, and AI-readable documentation links for company domains.

LLMs.txt AI Discovery Auditor

glowing_glove/llms-txt-ai-discovery-auditor

Check websites for llms.txt, llms-full.txt, robots AI crawler rules, sitemap hints, and AI search readiness gaps.

AI Bot & LLM Crawler Exposure Auditor

glowing_glove/ai-bot-llm-crawler-exposure-auditor

Audit robots.txt, llms.txt, AI crawler directives, public content exposure, and policy gaps for websites in the AI search era.

Generate an llms.txt draft and priority AI-readable link inventory from public sitemaps.

llms.txt Generator — Make Any Website AI-Readable

darknezz/llms-txt-generator

Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output for AI readiness, SEO for AI, docs portals and archiving. Agent-ready via Apify API and MCP.

LLMs.txt Scraper - AI Documentation Index Data

benthepythondev/llms-txt-scraper

Scrape llms.txt files and extract AI-readable documentation links, headings, titles and source metadata.

Check robots.txt, llms.txt, sitemap hints, and AI crawler blocking rules for websites and domains.

llms.txt Generator: llms-full.txt for Any Website (AI SEO)

fayoussef/llms-txt-generator

Generate a spec-compliant llms.txt and llms-full.txt for any website in one run. Crawls your sitemap or internal links, extracts every page's title, meta description and full Markdown content, and outputs two ready-to-upload files for ChatGPT, Claude, Perplexity and Gemini.

Audit domains for llms.txt, robots.txt AI bot rules, sitemap hints, and public AI crawler readiness signals.

llms.txt Generator: Make Any Website AI-Readable

apifmcpfactory/llms-txt-generator

Generates a spec-compliant llms.txt file for any website from the site's own pages and metadata (no AI invention) - the fix when an AI-readiness audit finds llms.txt missing; $0.02 per site.