Pricing

from $1.00 / 1,000 results

Try for free

Go to Apify Store

llms.txt Reader: Structured Site Maps for AI Agents

Try for free

Check up to 100 sites for llms.txt and get the parsed structure back as JSON: title, summary, sections and every link with its notes. Sites without the file return not-found so you can measure adoption across a list. Built for AI agents and pipelines that want machine readable site maps.

Pricing

from $1.00 / 1,000 results

Rating

0.0

(0)

Developer

Stive Hassan

Actor stats

Bookmarked

Total users

Monthly active users

11 days ago

Last modified

llms.txt Reader

Check any list of sites for llms.txt and get the parsed structure back as JSON: title, summary, sections, and every link with its notes. Built for AI agents and pipelines that want machine-readable site maps without writing a parser.

What it does / What it does NOT do

Fetches /llms.txt (and optionally /llms-full.txt) for up to 100 sites per run, validates the format, and outputs one structured record per file. Sites without the file come back as status not-found, so you can measure llms.txt adoption across a list.
Not supported: crawling pages, logins, or bypassing blocks. Blocked sites are reported as status blocked, never guessed.

Input / Output

Input: sites, tryFull. Output: one dataset record per site/file with status ok | not-found | not-llms-txt | blocked | error.

Pricing

Pay-per-event, per site checked. Free-plan runs are not charged by Apify.

Reliability

Fixture-tested nightly; live-verified weekly. Missing data is null, never fabricated.

LLMs.txt Discovery

toronto_777/llms-txt-discovery

Find public llms.txt, llms-full.txt, and AI-readable documentation links for company domains.

Steven Feng

LLMs.txt Checker — AI Readiness Monitor

q_services/llms-txt-checker

Check websites for llms.txt and llms-full.txt, extract sections and links for AI readiness and GEO audits.

Q Services

llms.txt Generator: Make Any Website AI-Readable

apifmcpfactory/llms-txt-generator

Generates a spec-compliant llms.txt file for any website from the site's own pages and metadata (no AI invention) - the fix when an AI-readiness audit finds llms.txt missing; $0.02 per site.

Howth Technology Factory

llms.txt Generator: Make Any Website LLM-Ready

f0rty7even/llms-txt-generator

Crawl a website and generate an llms.txt file (the emerging standard that helps LLMs and AI agents understand your site). Optionally build llms-full.txt with full page content. Output saved to the key-value store.

Michael Yousrie

LLMs.txt AI Discovery Auditor

glowing_glove/llms-txt-ai-discovery-auditor

Check websites for llms.txt, llms-full.txt, robots AI crawler rules, sitemap hints, and AI search readiness gaps.

Ushba Khan

llms.txt + llms-full.txt Generator for Any Docs Site

ianymu/llms-txt-converter

llms-txt-converter is an Apify Actor that crawls any documentation site and outputs the two GEO-standard files (llms.txt and llms-full.txt) used by ChatGPT, Claude, Perplexity, Gemini, and Bing Copilot to find and cite your docs.

Yanlong Mu

llms.txt Generator — Make Any Website AI-Readable

darknezz/llms-txt-generator

Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output for AI readiness, SEO for AI, docs portals and archiving. Agent-ready via Apify API and MCP.

Oaida Adrian

llms.txt Generator - Make Your Site AI-Discoverable

dltik/llms-txt-generator

Generate a standards-compliant llms.txt file for any website. Crawls your sitemap, extracts page titles & descriptions, groups them into sections, and outputs a clean llms.txt that helps ChatGPT, Claude, Gemini & Perplexity understand your site. Optional AI enrichment.

Walid

LLMs.txt Discovery Auditor

phoenix2810/llms-txt-discovery-auditor

Audit a website for /llms.txt and /llms-full.txt AI discovery readiness. Get a readiness score, markdown structure summary, and actionable recommendations. Built for GEO/AI-SEO consultants, documentation teams, and devrel.

Sanskar Jaiswal

🤖 llms.txt Generator & Auditor - Make Sites AI-Readable

that_red_bird/llms-txt-generator

⚡ Generate a spec-shaped llms.txt for any site in one run, and audit an existing one. ✅ Sitemap discovery, clean markdown extraction, optional llms-full.txt. ✅ Coverage %, missing pages, stale URLs and spec issues. Reads only the site itself — nothing to block it.