AI Crawler Access Checker

AI Crawler Checker – robots.txt Rules for GPTBot & ClaudeBot

locomint/ai-crawler-checker

AI crawler checker: read each website's robots.txt and see whether GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot and 13 other AI crawlers are allowed or blocked, with the line that decided each verdict, plus Content-Signal and llms.txt.

Hreflang Tag Checker API | SEO Validator

andok/hreflang-checker

Bulk validate website hreflang tags for international SEO. Audit multi-language sites for missing x-default tags, bidirectional errors, and ISO code compliance.

HTTP Status Code Checker & Bulk URL Checker API

pink_comic/http-status-code-checker

Bulk URL checker and HTTP status scraper for SEO audits. Check status codes, 301/302 redirect chains, response times, content types, and broken URLs. Use the API for website status checks, migrations, monitoring workflows, and link QA.

ChatGPT Ads & AI Crawler Policy Auditor

x2ppi/ai-crawler-readiness-auditor

ChatGPT Ads landing-page policy audit: bulk-check path-specific OAI-AdsBot access, AI bot rules, llms.txt, sitemap, and metadata.

Robots AI Crawler Policy Checker

seeb/robots-ai-crawler-policy-checker

Analyze robots.txt and crawler policy text for AI bot access rules, disallow paths, sitemap references, and blocked model-training crawlers.

Audit which AI crawlers can access a site via robots.txt, llms.txt, and sitemap signals for GPTBot, ClaudeBot, PerplexityBot, and more.

llms.txt Checker: AI Search Readiness Audit

apifmcpfactory/llms-txt-checker

Audits whether a website is readable by AI assistants (llms.txt, robots.txt AI-crawler access, sitemap) and returns a 0-100 score with concrete fixes - use when asked whether a site is visible to ChatGPT/Claude or how to improve that; $0.005 per site audited.

Catch SEO disasters before Google does: watches robots.txt, XML sitemaps and key pages for Disallow: /, noindex, canonical and status changes, sitemap URL drops and errors. Diffs vs last run, Slack/webhook alerts, HTML report.

Fetch a public website's robots.txt and report which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Bytespider, and more) are blocked, allowed, or defaulted. Returns an AI-visibility readiness score with recommendations for GEO and AI-SEO strategy.

AI Search Visibility Checker

seeb/ai-search-visibility-checker

Check public pages and search-style result pages for target brand, domain, topic, and competitor visibility signals.