llms.txt Generator & Checker: AI Search Readiness Audit

Generate, audit and monitor your llms.txt so ChatGPT and AI agents can read your site. Get an AI-readiness score and a ready-to-publish file.

Works on public pages only. We store nothing personal.

One Actor, three modes, picked with a single mode field: audit a site's current llms.txt, generate a ready-to-publish one from the site's own pages, or monitor it over time and get alerted when something breaks.

Use with AI agents (MCP)

This Actor is also an MCP tool — AI agents like Claude and Cursor can call it directly; usage bills through your Apify account.

MCP server URL:

https://mcp.apify.com?tools=apifmcpfactory/llms-txt-suite

Claude Desktop (claude_desktop_config.json):

{
    "mcpServers": {
        "llms-txt-suite": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com?tools=apifmcpfactory/llms-txt-suite",
                "--header",
                "Authorization: Bearer YOUR_APIFY_TOKEN"
            ]
        }
    }
}

Cursor (.cursor/mcp.json):

{
    "mcpServers": {
        "llms-txt-suite": {
            "url": "https://mcp.apify.com?tools=apifmcpfactory/llms-txt-suite",
            "headers": {
                "Authorization": "Bearer YOUR_APIFY_TOKEN"
            }
        }
    }
}

Replace YOUR_APIFY_TOKEN with your API token from Apify Console.

No Apify account? AI agents can also pay per call with USDC via the x402 protocol: buy a prepaid token at agi.apify.com and use it as the Bearer token above.

One tool, three actions

An agent (or you) picks the action with the mode field. Each mode has one job:

Mode	Verb + object + result	Example call
audit	Score a site's llms.txt AI-readiness (0-100) and list concrete fixes.	{ "mode": "audit", "siteUrl": "https://docs.stripe.com" }
generate	Draft a spec-compliant llms.txt (and optionally llms-full.txt) from the site's own pages.	{ "mode": "generate", "siteUrl": "https://example.com", "maxPages": 50 }
monitor	Re-audit a site and diff the result against its last check — pair with an Apify Schedule.	{ "mode": "monitor", "siteUrl": "https://example.com" }

What does llms.txt Generator & Checker do?

llms.txt is the emerging standard that gives AI assistants — ChatGPT, Claude, Perplexity — a curated, machine-readable guide to a website. This Actor covers the full lifecycle in one tool instead of three separate ones:

1. Audit — fetches /llms.txt (and /llms-full.txt if present), checks it's present, well-formed, has a title and summary, has sectioned links, and that those links still resolve, then returns a 0-100 AI-readiness score with plain-English recommendations.
2. Generate — reads the site's sitemap.xml first (falling back to a shallow crawl of homepage links, capped at maxPages), and assembles a spec-compliant llms.txt — title, summary, and sectioned [page](url): description links — from the site's own titles and meta descriptions. Nothing is invented by AI. Optionally also builds llms-full.txt with each page's body content.
3. Monitor — re-runs the audit and diffs the score, content hash, and check results against the last run, persisted per site. Attach this mode to an Apify Schedule to get notified the moment a site's llms.txt disappears, breaks, or goes stale.

Why use it?

• One tool covers the whole loop — audit finds the gap, generate fixes it, monitor keeps it fixed. No juggling three separate Actors.
• SEO / GEO agencies — run audit across a client list for a scored, explainable report; run generate to hand over the fix; schedule monitor to prove ongoing value.
• Site owners — one-click check of your own AI visibility, and a file you can publish immediately.

How to use it

1. Open the Input tab and pick a mode (defaults to audit).
2. Paste your Website URL (a working example is pre-filled — just click Start to try it).
3. Leave the advanced options at their defaults, or tune maxPages, includeFullText, validateLinks, respectRobots.
4. Click Start. Results appear in the Output tab — a scored table for audit/monitor, or a download link for the generated file.
5. For ongoing monitoring, click Schedule on this Actor, set mode to monitor, and pick an interval (e.g. weekly).

Input

{
    "mode": "audit",
    "siteUrl": "https://docs.stripe.com",
    "maxPages": 50,
    "includeFullText": false,
    "validateLinks": true,
    "respectRobots": true
}

Field	Used by	Description
mode	all	audit | generate | monitor. Defaults to audit.
siteUrl	all	Site or direct llms.txt URL. Bare domains are accepted.
maxPages	generate	Crawl cap (1-200, default 50) — also the cost guardrail.
includeFullText	generate	Also build llms-full.txt from page body content.
validateLinks	audit, monitor	Sample-check that llms.txt links still resolve.
respectRobots	all	Honor the site's robots.txt while crawling or validating links.

Output

Audit (and the audit half of monitor):

{
    "mode": "audit",
    "siteUrl": "https://docs.stripe.com",
    "hasLlmsTxt": true,
    "score": 85,
    "checks": [
        { "name": "present", "passed": true, "detail": "Found at https://docs.stripe.com/llms.txt." },
        { "name": "wellFormed", "passed": true, "detail": "Starts with a \"# Title\" heading and is a reasonable size." },
        { "name": "hasTitleSummary", "passed": true, "detail": "Has both a title and a \"> summary\" blockquote." },
        { "name": "hasSectionedLinks", "passed": true, "detail": "6 section(s), 214 link(s)." },
        { "name": "linksResolve", "passed": false, "detail": "17/20 sampled link(s) resolve. Broken: https://docs.stripe.com/old-page." },
        { "name": "notStale", "passed": true, "detail": "Link rot is within a normal range." }
    ],
    "missingSections": ["llms-full.txt (full-content companion file)"],
    "recommendations": ["Fix or remove 3 broken link(s): https://docs.stripe.com/old-page."]
}

Generate:

{
    "mode": "generate",
    "siteUrl": "https://example.com",
    "llmsTxt": "# Example\n\n> Website at example.com.\n\n## Pages\n\n- [About](https://example.com/about)\n...",
    "pagesIncluded": ["https://example.com/about", "https://example.com/pricing"],
    "fileUrl": "https://api.apify.com/v2/key-value-stores/.../records/example.com-llms.txt"
}

Monitor:

{
    "mode": "monitor",
    "siteUrl": "https://example.com",
    "previousScore": 90,
    "currentScore": 55,
    "changed": true,
    "diff": ["Score changed from 90 to 55 (-35).", "llms.txt content changed since the last check."],
    "status": "changed"
}

Every run also stamps its raw run duration (ms) and, on failure, a plain error field — never a raw crash.

Pricing — pay per event

Event	Price	Charged when
Audit run	$0.01	one completed audit of one site
Generate run	$0.03	one completed llms.txt draft of one site
Monitor run	$0.005	one completed monitor check of one site

Only the mode that actually ran is charged, and only on success — a failed run (bad URL, unreachable site) costs nothing. maxPages bounds generate's crawl so a single run's compute never runs away, keeping a comfortable margin under every price above even for a full 200-page crawl.

Compliance

Public pages only — this Actor never accesses anything behind a login or paywall. respectRobots (on by default) makes every mode honor a site's robots.txt. Nothing personal is stored: the only state kept between runs is the Monitor baseline (a score and a content hash, keyed by site URL), and that's derived entirely from the site's own public llms.txt.

FAQ

What is llms.txt? A proposed standard (llmstxt.org) — a markdown file at /llms.txt giving AI assistants a curated guide to a site, like robots.txt for the AI era.

Do I need to run audit before generate? No — generate works standalone. But running audit first tells you why you need it, and running it again after publishing confirms the fix worked.

How does monitor remember the last score? It stores a small baseline (score + content hash) per site URL in a persistent key-value store — no page content, no personal data. Attach monitor to an Apify Schedule and each scheduled run compares against the previous one automatically.

Is the generated file really not AI-written? Correct — every line comes from the site's own titles, meta descriptions, and (for llms-full.txt) body text. Deterministic in, deterministic out.

More tools from MCP Factory

• llms.txt Checker — a lighter, robots.txt-focused AI-crawler-access audit (GPTBot, ClaudeBot, PerplexityBot access status).
• llms.txt Generator — the standalone generator this Actor's generate mode builds on.
• Tech Stack Detector — find out what any website is built with (CMS, ecommerce, frameworks, analytics).
• EU VAT Validator — bulk-validate EU VAT numbers against the EU's official VIES service.
• PDF Toolkit — merge, split, compress, convert, rotate and watermark PDFs.