AI Search Readiness Audit - llms.txt, AI Bots & Schema

GEO / AEO Website Audit — AI Search Readiness

rtworule/ai-search-readiness-auditor

Audit public websites for GEO and AEO readiness: AI crawler access, robots.txt, llms.txt, sitemaps, Schema.org, social metadata, scores, and prioritized fixes.

AI Crawler Robots.txt Audit & llms.txt Checker

davidbenittah/ai-crawler-robots-audit

Check robots.txt and llms.txt on up to 50 sites and see which AI crawlers are blocked: OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, GPTBot and ClaudeBot. Returns a 0-100 AI search access score, the rule matched per bot, and a paste-ready robots.txt fix.

AI Search Visibility Tracker — ChatGPT, Gemini & More

cheapapi/ai-search-visibility-tracker

Check if ChatGPT, Gemini, Claude & Perplexity mention your brand: position, sentiment, cited sources, competitors & share of voice. No API keys.

Score any list of websites 0-100 on how well ChatGPT, Claude, Perplexity and Gemini agents can read them: robots.txt AI-bot blocks, firewall blocks, markdown negotiation, llms.txt, sitemap, JS-only pages, schema, hidden FAQs. Lead flag + pitch line per site.

Robots Txt Audit

apage/robots-txt-audit

Audit robots.txt files for AI crawler access. Get an AI Readiness Score (0-100), analyze 61+ AI crawlers (ChatGPT, Claude, Perplexity, Gemini), detect syntax errors, security concerns, and get actionable recommendations. Batch audit multiple domains at once with optional subdomain scanning.

Website to RAG Chunks - Re-crawl and Update Only What Changed

that_red_bird/website-to-rag-incremental

Stop re-embedding your whole site. Crawl docs or any site into heading-aware, token-counted RAG chunks; on re-runs get only new and changed chunks plus the ids to delete from your vector DB. Sitemap and robots.txt aware. JSON, JSONL, Markdown. No embedding or API key.

Audit which AI crawlers can access a site via robots.txt, llms.txt, and sitemap signals for GPTBot, ClaudeBot, PerplexityBot, and more.

npm & PyPI Package Scraper - Downloads & Risk Flags

that_red_bird/package-intel

Audit open-source dependencies fast. Profile any npm or PyPI package from the official registries: downloads, versions, maintainers, licence, freshness, deprecation, plus risk flags (DEPRECATED, UNMAINTAINED_2Y, SINGLE_MAINTAINER, NO_LICENSE) for supply-chain checks.

Podcast Transcript & Episode Scraper - From RSS, No Audio

that_red_bird/podcast-transcript-scraper

Get podcast episodes with the transcripts publishers make public: reads RSS podcast:transcript files (SRT, VTT, JSON, HTML), transcript links in show notes, chapters, guests, sponsors and links. Search keywords across episodes. Not speech-to-text, no API key.

SEO Audit Crawler - Find What Broke Since Last Crawl

that_red_bird/seo-audit-regression-crawler

Find what broke on your site since the last crawl: pages that now 404, new noindex, changed canonicals, titles, H1s and schema, redirect chains and broken internal links, ranked by severity with a plain-English summary and Slack/Discord alert. Re-run after every release.