Pricing

from $17.00 / 1,000 ai crawler policy audits

AI Crawler Access Checker

Audit up to 100 sites for 16 AI crawler policies. Get search vs training verdicts, robots.txt and llms.txt evidence, confidence, unknown states, priorities, and next actions for GEO/AEO reviews.

Pricing

from $17.00 / 1,000 ai crawler policy audits

Rating

0.0

(0)

Developer

Tim Zinin

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

AI Crawler Access Checker — robots.txt & llms.txt Audit API through CLI

The Apify CLI is the official tool that allows you to use AI Crawler Access Checker locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

AI Crawler Access Checker API in Python

AI Crawler Access Checker API in JavaScript

AI Crawler Access Checker OpenAPI definition

AI Crawler Access Checker API

llms.txt Auditor & AI Crawler Policy Checker

zinin/llms-txt-auditor

Audit public llms.txt, llms-full.txt, and root robots.txt rules for nine named AI crawlers. Get evidence-backed presence, policy, confidence, gaps, and review actions for each complete domain audit.

Tim Zinin

GEO / AEO Website Audit — AI Search Readiness

rtworule/ai-search-readiness-auditor

Audit public websites for GEO and AEO readiness: AI crawler access, robots.txt, llms.txt, sitemaps, Schema.org, social metadata, scores, and prioritized fixes.

Kunteper Koyu

AI Crawler Robots.txt Audit & llms.txt Checker

davidbenittah/ai-crawler-robots-audit

Check robots.txt and llms.txt on up to 50 sites and see which AI crawlers are blocked: OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, GPTBot and ClaudeBot. Returns a 0-100 AI search access score, the rule matched per bot, and a paste-ready robots.txt fix.

David

AI Training Blocker Checker

bipbip-apis/ai-training-blocker-checker

Check robots.txt, llms.txt, sitemap hints, and AI crawler blocking rules for websites and domains.

Daniel Christensen

AI Crawler Readiness Auditor

nectared_screen/ai-crawler-readiness-auditor

Audit domains for llms.txt, robots.txt AI bot rules, sitemap hints, and public AI crawler readiness signals.

jaehan byun

LLMs.txt AI Discovery Auditor

glowing_glove/llms-txt-ai-discovery-auditor

Check websites for llms.txt, llms-full.txt, robots AI crawler rules, sitemap hints, and AI search readiness gaps.

Ushba Khan

LLMs.txt / AI Readiness Auditor

civicdataworks/llms-txt-ai-readiness-auditor

Audit websites for AI crawler/readiness basics: llms.txt, sitemap, robots rules, titles, metadata, and AI crawler directives.

Rowan Mercer

AI Visibility Site Audit — robots.txt & llms.txt checks

43sunsets/ai-visibility-site-audit

Evidence-based audit of your own website: which AI crawlers robots.txt lets in (RFC 9309, 21 user-agent and control tokens, checked against the operators' own docs), which pages are discoverable, and each page's technical readiness — a ticket-grade fix plan with honest coverage disclosure.

Ken Yao

AI Bot & LLM Crawler Exposure Auditor

glowing_glove/ai-bot-llm-crawler-exposure-auditor

Audit robots.txt, llms.txt, AI crawler directives, public content exposure, and policy gaps for websites in the AI search era.

Ushba Khan

SEO, GEO & AEO Audit — AI Search Readiness Checker

fayoussef/seo-geo-aeo-audit

Audit any website for Google SEO and AI search visibility in one run. Get 0–100 SEO, GEO & AEO scores per page, AI-crawler & llms.txt checks, schema validation, and a prioritized fix list with a shareable HTML report. No API keys needed — just enter a URL.