AI Crawler Checker: GPTBot, ClaudeBot & Cloudflare Blocks

AI Crawler Access Checker

zinin/ai-crawler-access-checker

Audit up to 100 sites for 16 AI crawler policies. Get search vs training verdicts, robots.txt and llms.txt evidence, confidence, unknown states, priorities, and next actions for GEO/AEO reviews.

GEO Audit - AI Search Optimization Checker

lofomachines/geo-technical-audit

Analyze your web pages for Generative Engine Optimization (GEO). Get actionable insights to improve visibility in ChatGPT, Perplexity, Gemini, Google AI Overviews, and other AI search engines. Check schema markup, semantic HTML, E-E-A-T signals, and content structure.

AI Readiness Auditor

rationalistic_counsel/ai-readiness-auditor

Check how AI-ready any website is. Get an AI Readiness Score (0-100) checking llms.txt, robots.txt AI crawler directives, Schema.org structured data, and meta tags. No API key needed.

AI Crawler Robots.txt Audit & llms.txt Checker

davidbenittah/ai-crawler-robots-audit

Check robots.txt and llms.txt on up to 50 sites and see which AI crawlers are blocked: OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, GPTBot and ClaudeBot. Returns a 0-100 AI search access score, the rule matched per bot, and a paste-ready robots.txt fix.

🔎 Crawl websites and turn technical SEO, accessibility, security, performance, broken-link, structured-data, robots.txt, llms.txt, and AI crawler signals into prioritized 0–100 audit scores and client-ready reports.

AI Crawler Checker - robots.txt Checker for GPTBot & AI Bots

tidytools/ai-crawler-access-checker

Check if GPTBot, ClaudeBot, PerplexityBot and 26 more AI crawlers are blocked on your site: bulk robots.txt check of up to 10,000 domains, AI search score, fix snippet, Content Signals, change alerts. $2/1k sites.

AI Brand Visibility Tracker — AI Overview & Perplexity

aitorsm/ai-visibility-tracker

Track whether ChatGPT, Google AI Overview, Perplexity, Gemini and Copilot mention your brand when buyers ask for a recommendation. Repeat checks give a real mention rate with a margin of error, plus who gets named instead, what changed since last week, and what each gap is worth.

Audit sitemap URLs, robots.txt, HTTP status, canonical/noindex signals, and AI crawler access as structured JSON for technical SEO and agents.

Audit robots.txt policies for GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot and other AI crawlers.

Audit which AI crawlers can access a site via robots.txt, llms.txt, and sitemap signals for GPTBot, ClaudeBot, PerplexityBot, and more.