AI Crawler Checker - robots.txt Checker for GPTBot & AI Bots

Broken Link Checker - Find Dead 404 Links

logiover/broken-link-checker

Site-wide broken link checker: crawl any website, find 404 and dead links, export the link audit to CSV or JSON with source page and status code.

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

AI Crawler Checker – robots.txt Rules for GPTBot & ClaudeBot

locomint/ai-crawler-checker

AI crawler checker: read each website's robots.txt and see whether GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot and 13 other AI crawlers are allowed or blocked, with the line that decided each verdict, plus Content-Signal and llms.txt.

AI Search Volume Explorer - Keyword Demand Across AI Search

santhej/ai-search-volume-explorer

Discover how often keywords are actually searched inside AI assistants like ChatGPT. Get AI search volume and 12-month trend for any keyword — the GEO/AEO metric traditional SEO tools can't show you. Clean JSON/CSV. Future-proof your content. No API keys.

Hreflang Tag Checker API | SEO Validator

andok/hreflang-checker

Bulk validate website hreflang tags for international SEO. Audit multi-language sites for missing x-default tags, bidirectional errors, and ISO code compliance.

Citation Checker AI

alizarin_refrigerator-owner/citation-checker-ai

Check business citations across 36+ directories. Verify NAP (Name, Address, Phone) consistency across major business directories. Essential for local SEO

Robots AI Crawler Policy Checker

seeb/robots-ai-crawler-policy-checker

Analyze robots.txt and crawler policy text for AI bot access rules, disallow paths, sitemap references, and blocked model-training crawlers.

Audit robots.txt policies for GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot and other AI crawlers.

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Fetch a public website's robots.txt and report which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Bytespider, and more) are blocked, allowed, or defaulted. Returns an AI-visibility readiness score with recommendations for GEO and AI-SEO strategy.