Pricing

from $8.50 / 1,000 complete domain audits

llms.txt Auditor & AI Crawler Policy Checker

Audit public llms.txt, llms-full.txt, and root robots.txt rules for nine named AI crawlers. Get evidence-backed presence, policy, confidence, gaps, and review actions for each complete domain audit.

Pricing

from $8.50 / 1,000 complete domain audits

Rating

0.0

(0)

Developer

Tim Zinin

Actor stats

Bookmarked

Total users

Monthly active users

13 days ago

Last modified

Categories

SEO tools

You can access the llms.txt Auditor & AI Crawler Policy Checker programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = { "items": [
9        "resend.com",
10        "upstash.com",
11    ] }
12
13# Run the Actor and wait for it to finish
14run = client.actor("zinin/llms-txt-auditor").call(run_input=run_input)
15
16# Fetch and print Actor results from the run's dataset (if there are any)
17print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
18for item in client.dataset(run.default_dataset_id).iterate_items():
19    print(item)
20
21# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

llms.txt Auditor & AI Crawler Policy Checker API in Python

The Apify API client for Python is the official library that allows you to use llms.txt Auditor & AI Crawler Policy Checker API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

llms.txt Auditor & AI Crawler Policy Checker API in JavaScript

llms.txt Auditor & AI Crawler Policy Checker API through CLI

llms.txt Auditor & AI Crawler Policy Checker OpenAPI definition

llms.txt Auditor & AI Crawler Policy Checker API

LLMs.txt AI Discovery Auditor

glowing_glove/llms-txt-ai-discovery-auditor

Check websites for llms.txt, llms-full.txt, robots AI crawler rules, sitemap hints, and AI search readiness gaps.

Ushba Khan

LLMs.txt Discovery

toronto_777/llms-txt-discovery

Find public llms.txt, llms-full.txt, and AI-readable documentation links for company domains.

Steven Feng

LLMS.txt Auditor

junipr/llms-txt-auditor

Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

junipr

LLMs.txt Checker — AI Readiness Monitor

q_services/llms-txt-checker

Check websites for llms.txt and llms-full.txt, extract sections and links for AI readiness and GEO audits.

Q Services

AI Crawler Policy Audit — robots.txt, llms.txt, noai

multiplex/ai-crawl-policy

See exactly which AI crawlers a site allows. Per domain it reads robots.txt for 25 named AI agents (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot…), plus ai.txt, llms.txt, noai meta, X-Robots-Tag and tdmrep.json, and quotes the rule behind every answer. Alerts on policy changes.

Daniel James

LLMs.txt / AI Readiness Auditor

civicdataworks/llms-txt-ai-readiness-auditor

Audit websites for AI crawler/readiness basics: llms.txt, sitemap, robots rules, titles, metadata, and AI crawler directives.

Rowan Mercer

AI SEO Audit: robots.txt, llms.txt & 26 AI Crawlers

aiqlabs/seo-ai-visibility-auditor

Crawls a site from one starting URL and reports how visible it is to AI: which AI crawlers robots.txt admits, whether llms.txt exists, and what structured data and metadata each page offers. Give it a domain to crawl - for a fixed list of pages, use SEO Audit Tool instead.

Ai-Q Labs

llms.txt Generator — Make Any Website AI-Readable

darknezz/llms-txt-generator

Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output for AI readiness, SEO for AI, docs portals and archiving. Agent-ready via Apify API and MCP.

Oaida Adrian

llms.txt + llms-full.txt Generator for Any Docs Site

ianymu/llms-txt-converter

llms-txt-converter is an Apify Actor that crawls any documentation site and outputs the two GEO-standard files (llms.txt and llms-full.txt) used by ChatGPT, Claude, Perplexity, Gemini, and Bing Copilot to find and cite your docs.