llms.txt Generator — make any website AI-readable

Generate a ready-to-publish llms.txt file for any website in one run. llms.txt (llmstxt.org) is the emerging standard that gives AI assistants a curated, machine-readable guide to your site — and the single biggest fix for AI visibility. This Actor reads a site's own pages and metadata and produces a spec-compliant llms.txt: site title, one-line summary, and organized sections of [page title](url): description links. Upload the result to /llms.txt on the site and you're done.

The companion fix to our llms.txt Checker: audit first, see "no llms.txt — the single biggest AI-readiness fix", then generate the fix here. Also works with our AI Visibility Checker ChatGPT app.

$0.02 per site plus a small per-run start fee. An SEO agency can fix a whole client list for pennies.

Honest by design — no AI invention

Every line of the generated file comes from the site's own content: page titles, meta descriptions, or the page's first paragraph. Nothing is written by an AI, so nothing can be made up. Deterministic in, deterministic out — the same site produces the same file.

How it works

1. Page discovery — reads the site's sitemap.xml first (including sitemap indexes and robots.txt Sitemap: declarations); if there is no sitemap, it falls back to the homepage's links.
2. Metadata extraction — up to maxPagesPerSite pages (default 30, cap 100) are fetched; each contributes its title and meta description (or first paragraph).
3. Section grouping — pages are grouped by URL path into Docs, Products & Services, Blog, Company, and Pages.
4. Spec-compliant assembly — H1 title, > blockquote summary from the homepage metadata, and section lists in the exact - [name](url): details format from llmstxt.org.

Input

{
    "urls": ["https://books.toscrape.com", "https://quotes.toscrape.com"],
    "maxPagesPerSite": 30
}

Output

One dataset item per site — llmsTxt holds the complete file text, ready to save as /llms.txt:

{
    "url": "https://books.toscrape.com",
    "origin": "https://books.toscrape.com",
    "llmsTxt": "# Books to Scrape - Sandbox\n\n> Website at books.toscrape.com.\n\nThis file lists the key pages of books.toscrape.com for AI assistants. It was generated from the site's own page titles and descriptions.\n\n## Products & Services\n\n- [Books](https://books.toscrape.com/catalogue/category/books_1/index.html)\n- [Travel](https://books.toscrape.com/catalogue/category/books/travel_2/index.html)\n...",
    "pagesIncluded": 30,
    "sectionsUsed": ["Products & Services", "Pages"],
    "generatedAt": "2026-07-20T16:04:11.000Z"
}

Unreachable sites produce an item with an error field — a batch never dies mid-run, and output always reconciles 1:1 with input.

Who is it for?

• Site owners who ran an AI-readiness audit and need the llms.txt fix now.
• SEO / GEO agencies fixing AI visibility for client lists in bulk — generate, review, publish.

Pricing

Pay per event: a small flat fee per run ($0.001) plus $0.02 per site generated. Comparable generators charge $0.05 flat or $0.01 per page; with the page cap keeping compute at a fraction of a cent, this stays the cheapest way to fix a whole portfolio.

FAQ

Is the output really spec-compliant? Yes — H1 title, blockquote summary, markdown sections, and - [name](url): details link lines exactly as llmstxt.org specifies.

What if the site has no sitemap? The generator falls back to the links on the homepage — usually the site's most important pages anyway.

Should I edit the file before publishing? You can publish it as-is, but a quick human pass never hurts: reorder sections, drop pages you don't want AI assistants to prioritize, or tighten descriptions.

How do I publish it? Save the llmsTxt text as a file named llms.txt in the root of your website (so it loads at https://yoursite.com/llms.txt), then re-run the llms.txt Checker to confirm your score improved.

More tools from MCP Factory

• llms.txt Checker — the audit side: llms.txt, AI crawler access in robots.txt, sitemap, and a 0-100 AI readiness score with fixes.
• Tech Stack Detector — find out what any website is built with (CMS, ecommerce, frameworks, analytics) at ~$5 per 1,000 sites.
• EU VAT Validator — bulk EU VAT validation against the EU's official VIES service, with official audit evidence.