llms.txt Generator — Make Your Site AI-Discoverable

⭐ Bookmark this actor so you can re-generate your llms.txt every time your site changes — it takes one click and keeps your AI discoverability fresh.

Generate a standards-compliant llms.txt file for any website in seconds. This actor crawls your sitemap (or homepage), extracts every page's title and description, groups pages into clean sections, and outputs a ready-to-host llms.txt that helps ChatGPT, Claude, Gemini, Perplexity and other AI assistants understand and cite your site.

llms.txt is the emerging standard (llmstxt.org) for telling Large Language Models what your site is about and which pages matter — the AI-era equivalent of robots.txt + sitemap.xml.

Why you need an llms.txt

AI assistants increasingly answer questions by reading the web. A clean llms.txt at https://yoursite.com/llms.txt:

• Tells LLMs what your site/company does in one sentence
• Points them to your most important pages with human-readable descriptions
• Improves how accurately AI tools summarize and cite you
• Is fast becoming a baseline GEO (Generative Engine Optimization) signal

What this actor does

1. Discovers your URLs — reads robots.txt + sitemap.xml (and sitemap indexes). No sitemap? It falls back to crawling links from your homepage.
2. Extracts metadata — page title (og:title / <title>), meta description, <h1>, and og:site_name.
3. Sections your site — groups pages by path (/blog, /docs, /pricing, …) into a tidy table of contents.
4. Optional AI enrichment — a single LLM pass rewrites the site summary, regroups pages into logical sections, and adds concise descriptions where they're missing.
5. Outputs llms.txt — the full file is returned in the llms_txt field, ready to copy to your site root.

Input

Field	Type	Description
url	string (required)	Website to process, e.g. https://stripe.com
maxPages	integer	Max pages to include (1–300, default 100)
includeSitemap	boolean	Discover URLs from robots.txt + sitemap.xml (default true)
crawlIfNoSitemap	boolean	Crawl homepage links when no sitemap is found (default true)
aiEnrich	boolean	Run AI enrichment for summary + smart sections (default false)
aiModel	string	OpenRouter model for enrichment (default google/gemini-2.0-flash-001)
siteName	string	Optional override for the llms.txt title
siteSummary	string	Optional override for the one-line summary

Example input

{
  "url": "https://stripe.com",
  "maxPages": 100,
  "aiEnrich": true
}

Output

One record per run:

{
  "site_name": "Stripe",
  "host": "stripe.com",
  "page_count": 87,
  "section_count": 6,
  "ai_enriched": true,
  "llms_txt": "# Stripe\n\n> Financial infrastructure for the internet.\n\n## Main\n- [Pricing](https://stripe.com/pricing): Simple, transparent pricing...\n..."
}

Copy the llms_txt value into a file named llms.txt and host it at the root of your domain (https://yoursite.com/llms.txt).

Pricing

Pay-per-event, no subscription:

• Actor start — $0.00005 per run
• Page processed — $0.005 per page included in the llms.txt
• AI enrichment — $0.05 per run (only when aiEnrich is enabled)

A typical 100-page site costs about $0.50 structurally, or $0.55 with AI enrichment.

Use cases

• SEO / GEO teams — ship llms.txt across a portfolio of client sites in bulk.
• Agencies — add AI discoverability as a productized deliverable.
• Founders / indie hackers — make your docs and product pages legible to AI assistants.
• Content sites — surface your best articles to LLM crawlers.

Next step: measure your AI visibility

Generating llms.txt is step one. To see how often AI assistants actually mention and cite your brand, run GEO Visibility Score — it computes a 0–100 GEO Score, mention rate, citation rate, and Share of Voice vs competitors across ChatGPT, Claude, Gemini and Perplexity.

---

Built by dltik. Pure HTTP — no browser, fast and cheap.