llms.txt Generator & Checker (plus llms-full.txt)

Make any website AI-ready, with no login, no browser and no code. Export to JSON, CSV or Excel, or use the API.

Make any website AI-ready: checks whether /llms.txt and /llms-full.txt exist and validates them against the llms.txt spec, then crawls the site and generates a well-structured llms.txt (title, summary, sectioned link list with descriptions) and an llms-full.txt with every page as Markdown, ready to upload. Great for docs sites, SaaS and agencies doing AI SEO / GEO.

Use cases

• Generate an llms.txt for your own or a client's site in one run
• Audit whether competitors' sites are AI-ready
• Package a docs site into one Markdown file for an LLM context window

Input

Field	Type	Notes
websites	array	Homepages or docs roots.
maxPagesPerSite	integer
useSitemap	boolean
buildFullText	boolean	All pages as one Markdown file.
mainContentOnly	boolean
includeLinks	boolean
respectRobots	boolean
maxConcurrency	integer

Example input

{
 "websites": [
  "https://therealtour360.com",
  "https://docs.anthropic.com",
  "nonexistent-domain-zzq81.com"
 ],
 "maxPagesPerSite": 12
}

Output

One dataset item per result. Main columns: website, siteName, pagesIncluded, existingLlmsTxt, llmsTxtUrl, llmsFullTxtUrl, llmsFullTxtTokenEstimate, recommendation, error. A RUN_SUMMARY record is saved to the default key-value store. See the sample output in the Output tab.

Example output

{
  "website": "https://therealtour360.com",
  "existingLlmsTxt": {
    "found": false,
    "httpStatus": 200,
    "note": "URL returns an HTML page, not a Markdown file"
  },
  "existingLlmsFullTxt": {
    "found": false,
    "httpStatus": 404
  },
  "llmsTxtUrl": "https://api.apify.com/v2/key-value-stores/xgqjg2VctotpkdWdF/records/therealtour360.com-llms.txt?signature=1...",
  "llmsFullTxtUrl": "https://api.apify.com/v2/key-value-stores/xgqjg2VctotpkdWdF/records/therealtour360.com-llms-full.txt?signat...",
  "llmsFullTxtSizeBytes": 68703,
  "llmsFullTxtTokenEstimate": 17150,
  "siteName": "Real Tour 360",
  "pagesIncluded": 12,
  "sections": 2,
  "generatedLlmsTxt": "# Real Tour 360\n\n> Real estate photography, video, drone and Zillow 3D tours in Southern Middle Tennessee a...",
  "recommendation": "No llms.txt found; upload the generated file to https://therealtour360.com/llms.txt",
  "generatedAt": "2026-10-12T04:44:10.731Z"
}

Pricing

Pay per result, with no monthly rental or subscription:

• llms.txt generated for a site: $0.02
• Page crawled and included: $0.0005

Example: 1,000 results cost about $20.00 in fees. Apify's free plan includes $5 of monthly credit, which is enough to test this Actor.

FAQ

Do I need an account, cookies or a login for the source site? No. llms.txt Generator & Checker uses public data sources only, so there is nothing to connect and no account to risk.

How much will a run cost? You pay only for results delivered (see Pricing above), plus Apify platform usage, which the free plan's monthly credit easily covers for test runs. Start with a small limit to see the exact cost per run.

Can I export to CSV, Excel or Google Sheets? Yes. Every run's dataset can be downloaded as JSON, CSV, Excel, XML or HTML, or pushed to Google Sheets, Make, Zapier, n8n, Slack or a webhook through Apify integrations.

Can I run it on a schedule or from my own code? Yes. Add an Apify Schedule (hourly, daily, weekly) or call it through the Apify API / Python & JavaScript clients. It also works with AI agents through the Apify MCP server.

What happens if an input is invalid or a source is down? The run keeps going. The bad input gets a row with an error field so you can see what failed, and you are not charged for results that were not delivered.

Is this legal to use? It collects publicly available data. You are responsible for using the output in line with the source's terms and privacy laws such as GDPR/CCPA; don't use it for spam.

Something missing or broken? Open an issue on the Issues tab. Feature requests are usually added within a few days.

Scheduling / monitoring

Create a Schedule (daily/weekly) and connect a webhook or Slack/email integration.

Fair use

Official public APIs only, no login. Do not use output for spam or harassment. Respect the data sources' rate limits and terms.