Pricing

from $4.00 / 1,000 site validateds

Try for free

Go to Apify Store

llms.txt Validator & Generator

Try for free

Validate /llms.txt against the llmstxt.org format with a 0-100 score, per-check pass/warn/fail, broken-link check and fix suggestions, or generate a draft llms.txt from a site's sitemap and page metadata.

Pricing

from $4.00 / 1,000 site validateds

Rating

0.0

(0)

Developer

AI Crawl Ready

Actor stats

Bookmarked

Total users

Monthly active users

3 hours ago

Last modified

Categories

SEO tools

Developer tools

Sites

startUrls

Required

Sites to check, as full URLs (https://example.com). Each entry is reduced to its origin, and /llms.txt at that origin is validated. To validate a file in a subpath, enter its full URL (e.g. https://example.com/docs/llms.txt). Maximum 500 sites per run; duplicates are skipped.

Type:array

Mode

mode

Optional

validate = check the existing llms.txt; generate = build a draft llms.txt from the sitemap and page metadata; both = do both.

Type:string

Default:both

Options:

validategenerateboth

Check that linked URLs resolve

checkLinks

Optional

Send a HEAD request (GET fallback) to links listed in llms.txt, up to the cap below. Links disallowed by the linked host's robots.txt are skipped.

Type:boolean

Default:true

Max links to check per site

maxLinksToCheck

Optional

Links are sampled round-robin across sections so each section is represented. 0 disables link checking.

Type:integer

Minimum:0

Maximum:200

Default:20

Also check /llms-full.txt

checkLlmsFullTxt

Optional

Fetch the sibling llms-full.txt and report whether it exists, its size, content type and whether it starts with an H1. (llms-full.txt is a common convention, not part of the llmstxt.org spec, so it does not affect the score.)

Type:boolean

Default:true

Include raw llms.txt text

includeRawLlmsTxt

Optional

Add the fetched llms.txt text (first 100 KB) to each dataset item.

Type:boolean

Default:false

Max pages to sample for generation

maxSitemapPages

Optional

How many pages (from sitemap.xml, found via robots.txt Sitemap lines or /sitemap.xml) are fetched to read their titles and meta descriptions. Pages disallowed by robots.txt or marked noindex are skipped.

Type:integer

Minimum:1

Maximum:100

Default:15

Delay between requests to the same host (ms)

requestDelayMs

Optional

The actor never sends more than one request at a time to a host, and waits at least this long between requests to it. A larger robots.txt Crawl-delay (capped at 10 s) is honoured.

Type:integer

Minimum:250

Maximum:30000

Default:1000

Sites processed in parallel

maxConcurrency

Optional

Different sites are processed in parallel; requests to any one host stay sequential.

Type:integer

Minimum:1

Maximum:10

Default:3

llms.txt Generator + Validator

checksmithcats/llms-txt-generator-validator

Creates llms.txt and robots.txt snippets from submitted resources and validates their basic shape before the user publishes any files.

Checksmith Cats

📄 llms.txt Generator & Validator — AI Search Visibility

upstanding_biobot/llms-txt-generator

Generate llms.txt files for AI search visibility. Fetches sitemap, validates structure, checks AI bot access (OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended). 2 blue ocean competitors only.

Alexander Maksimchuk

LLMs.txt Generator

onescales/the-llms-txt-generator

The most powerful LLMs.txt Generator tool online. Generates LLMs.txt , llms-full.txt and markdown .md files within seconds! Get your website discovered, and recommended by ChatGPT, Claude, Google Gemini, Perplexity, Grok, and every AI. (Great for AEO, AIO, GEO, AI SEO) Made by Hi LLMs

One Scales

129

5.0

(2)

llms.txt Checker, Generator & Monitor: AI Search Readiness.

apifmcpfactory/llms-txt-suite

Generate, audit and monitor your llms.txt so ChatGPT and AI agents can read your site. Get an AI-readiness score and a ready-to-publish file. Works on public pages only. We store nothing personal.

Howth Technology Factory

AI Crawler Checker – robots.txt Rules for GPTBot & ClaudeBot

locomint/ai-crawler-checker

AI crawler checker: read each website's robots.txt and see whether GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot and 13 other AI crawlers are allowed or blocked, with the line that decided each verdict, plus Content-Signal and llms.txt.

Locomint

Structured Data Audit (JSON-LD / schema.org)

conserving_mastodon/structured-data-audit

Extract and validate a page's JSON-LD, report the schema.org types present, and flag missing recommended properties that block rich results and AI grounding. Detects microdata, RDFa, and Open Graph too. Charged only on completed audits.

Chris Arsenault

🩺 Website Health Check — All-in-One Audit

upstanding_biobot/website-health-check

Comprehensive website health audit: accessibility (axe-core), security headers, SSL/TLS, broken links, sitemap, robots.txt, structured data, meta tags, performance signals. One scan, full picture.

Alexander Maksimchuk

Website Tech Stack Detector

webdatatools/tech-stack-detector

Find out which CMS, e-commerce platform, analytics, ad pixels, chat widget and payment provider any website runs — one clean row per domain.

Murat Uzun

Sanctions Screening Suite: OFAC, EU, UK, UN & PEP Watchlists

apifmcpfactory/sanctions-screening

All-in-one sanctions screening suite for teams and AI agents. Screen, monitor and export against official OFAC, EU, UK OFSI, UN and OpenSanctions PEP watchlists. Get a CLEAR/REVIEW/ESCALATE verdict, auto-cleared false positives, an audit PDF and crypto-wallet screening. Official data, nothing stored

Howth Technology Factory

5.0

(1)

AI Crawler Checker & llms.txt Generator — Website AI Audit

dev-hoss/ai-search-readiness-audit

Check if AI search engines can crawl and cite your website: robots.txt rules for GPTBot, ClaudeBot, PerplexityBot and 17 more AI crawlers, llms.txt validation, sitemap, schema.org, meta tags and content extractability. 0-100 score, prioritized fixes, ready llms.txt.

Hossam Mohamed

llms.txt Generator: Make Any Website LLM-Ready

f0rty7even/llms-txt-generator

Crawl a website and generate an llms.txt file (the emerging standard that helps LLMs and AI agents understand your site). Optionally build llms-full.txt with full page content. Output saved to the key-value store.