llms.txt Generator – Make Any Website AI-Readable

Generate an llms.txt file for any website from its sitemap, grouped by section, and check whether one already exists. Copy-paste ready for AI search and agents.

Why this actor

• Only actor that builds llms.txt from sitemaps automatically; also detects existing files.
• Pay only for results: $20.00 per 1,000 sites, no start fee, failed lookups are free.
• Robust: automatic retries with exponential backoff, respects Retry-After, per-item error rows instead of crashed runs.
• Clean, flat JSON/CSV/Excel output ready for Google Sheets, Make, Zapier, n8n, Clay, or the Apify API.

Use cases

• Agencies offering AI-search readiness
• Site owners preparing for AI crawlers
• Audit competitors' AI readiness

How to use llms.txt Generator

1. Configure urls, maxLinks in the Input tab (or leave the prefilled sample values to test immediately).
2. Optionally set maxItems to cap the number of results and control spend.
3. Click Start and export structured results as JSON, CSV, Excel, or connect directly via API / MCP / n8n / Make.

Input

Field	Type	Description
urls	array	Sites to generate an llms.txt for (uses their sitemap).
maxLinks	integer	Links to include in llms.txt.
maxItems	integer	0 = no limit. Caps cost.

Example input:

{
  "urls": [
    "https://www.python.org"
  ]
}

Output

Each dataset item looks like this (real sample from a test run, long fields trimmed):

{
  "url": "https://www.python.org",
  "siteName": "Python.org",
  "existingLlmsTxt": false,
  "sitemapUrls": 0,
  "sections": 0,
  "linksIncluded": 0,
  "llmsTxt": "# Python.org\n\n> The official home of the Python Programming Language\n",
  "generatedAt": "2026-10-14T23:42:15+00:00"
}

Rows that could not be processed contain an error field and are not charged.

Extracted output fields

Every dataset row includes structured fields ready for spreadsheets, CRMs, and AI pipelines:

• Fields: url, siteName, existingLlmsTxt, sitemapUrls, sections, linksIncluded, llmsTxt, generatedAt

Comparison: llms.txt Generator vs alternatives

Feature	This Actor (shawndunn/llms-txt-generator)	Typical alternatives
Pricing model	$20 / 1,000 results, $0.00 start fee	$20–$49/mo subscription or per-run start fees
Failed / error lookups	$0.00 (never charged)	Often billed by compute time even on failures
API keys / proxies required	None (works out of the box)	Often requires paid third-party keys or residential proxies
AI Agent & MCP ready	Yes (mcp.apify.com compatible, optional defaults)	Strict required schemas that reject agent calls

Pricing

Pay-per-event: $0.02 per site (= $20.00 / 1,000). Use Max results to cap spend. Apify's free plan credit covers thousands of results per month.

Integrations & API

Run on a schedule in Apify Console, call via the Apify API / Python & JS clients, or connect to Make, Zapier, n8n, Google Sheets, Slack and webhooks. Works as a tool for AI agents via Apify MCP.

Use with AI agents (MCP)

llms.txt Generator is configured for zero-config AI agent tool use via the Apify MCP Server (no required-field blocking on defaulted inputs).

Streamable HTTP endpoint:

https://mcp.apify.com/?tools=shawndunn/llms-txt-generator

Claude Desktop / Cursor / Windsurf mcpServers config:

{
  "mcpServers": {
    "apify-llms-txt-generator": {
      "url": "https://mcp.apify.com/?tools=shawndunn/llms-txt-generator",
      "headers": {
        "Authorization": "Bearer <YOUR_APIFY_TOKEN>"
      }
    }
  }
}

Direct synchronous API call:

POST https://api.apify.com/v2/acts/shawndunn~llms-txt-generator/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>

FAQ

Is this an official tool? No. It is an independent actor and is not affiliated with, sponsored or endorsed by the data source or any trademark owner named here.

Is it legal? The actor only reads publicly available data from public endpoints, without logins, and does not bypass paywalls or access controls. You are responsible for using the data in line with applicable laws (e.g. GDPR) and the source sites' terms.

Something broke? Open an issue in the Issues tab; we typically respond within 24 hours and ship fixes fast.

More tools by shawndunn

• Bulk PageSpeed Insights & Core Web Vitals Checker (Lighthouse) — Audit hundreds of URLs with Google PageSpeed Insights: performance, SEO, accessibility and best-practices scores, LCP, CLS, TBT ($3/1k, no start fee).
• Sitemap URL Extractor — Extract every URL from any website's XML sitemaps: auto-discovery via robots.
• Broken Link Checker & Bulk URL Status Checker — Crawl a website and test every internal and external link, or bulk-check a list of URLs (url, statusCode, ok, finalUrl) ($1/1k, no start fee).
• Hugging Face Hub Explorer: Models, Datasets & Spaces — List and search Hugging Face models, datasets or Spaces with downloads, likes, trending score, pipeline task, library, license ($0.
• Wikipedia Article Extractor: Clean Text, Categories, Wikidata ID — Get clean plain text (intro or full article) for Wikipedia articles in any language, plus categories, Wikidata ID, short ($0.