AI Search Readiness Audit (GEO / AEO)

AI Crawler Checker & GEO / AEO Audit: check whether ChatGPT, Perplexity, Gemini and Claude can crawl and cite your website. For each site you get a 0-100 GEO score, an A-F grade and a plain list of fixes.

What you get

• Which AI search crawlers and AI training crawlers your robots.txt blocks.
• Whether llms.txt, a sitemap and JSON-LD structured data are present and valid.
• Page-level basics that answer engines rely on: title, meta description, canonical, headings, Open Graph, noindex.
• A prioritised fix list, one row per site, as JSON, CSV or Excel.

Use cases

• Agency monthly audits: run the same client list every month and compare scores.
• Competitor comparison: audit your site and five competitors in one run.
• Pre-launch check: confirm a new site is not blocking AI crawlers by mistake.
• GEO / AEO reporting: add a repeatable readiness score to SEO reports.

Quick start

Open the Input tab, keep the example sites (apify.com, github.com, wikipedia.org) or enter your own, and press Start. The Output tab shows one row per site.

What is checked

• robots.txt: exists; does not block everything; which AI crawlers are blocked. AI search bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, ClaudeBot, Claude-SearchBot, Google-Extended, Applebot-Extended) lower the score when blocked. AI training bots (GPTBot, anthropic-ai, CCBot, Bytespider, Meta-ExternalAgent) are reported but never scored: opting out of training is a legitimate choice.
• llms.txt: present, starts with an H1, number of links.
• sitemap: from robots.txt Sitemap: lines, or /sitemap.xml; URL count.
• The given page: title, meta description, canonical, lang, single <h1>, JSON-LD types (and invalid blocks), Open Graph, noindex (meta or X-Robots-Tag), amount of text in the raw HTML.

Input

{ "urls": ["https://apify.com", "https://github.com", "https://www.wikipedia.org"], "maxSites": 3 }

One audit per site (origin); duplicates are merged.

Output (one row per site)

origin, status (ok / unreachable), score, grade, robotsTxt, llmsTxt, sitemap, page, issues (list of fixes), error.

Pricing

Pay per event: one site-audited event per site with status: ok. Unreachable sites are not charged.

Use with AI agents (API)

Call the Actor through the Apify API, the Apify client libraries or an MCP client and read the dataset rows as JSON. Each row has a numeric score and an issues list, which an agent can turn into a to-do list.

FAQ

Does blocking GPTBot hurt my score? No. AI training bots are reported but never scored; opting out of training is a legitimate choice. Only AI search bots (OAI-SearchBot, PerplexityBot, ClaudeBot and similar) lower the score when blocked.

Is the score a ranking or citation measurement? No. It is a checklist with fixed weights, not a measurement of how often AI tools cite you.

Why does my JavaScript-only content count as missing? The HTML is not rendered, which is also how many AI crawlers see a page.

Am I charged for unreachable sites? No. Only sites with status: ok are charged.

Behaviour and limits

• Reads only public files and the one page you give. robots.txt is respected: if it disallows this Actor's user agent for your page, the page checks are skipped and said so.
• 1 second pause between sites. Pages are read up to 2 MB. HTML is not rendered: content that only appears after JavaScript runs counts as missing, which is also how many AI crawlers see it.
• The score is a checklist with fixed weights, not a measurement of rankings or citations.

Run the tests

python -m unittest discover -s tests -t . (no network access needed).

Related actors

• Video Frame Extractor: frames, scene changes and thumbnails from video file URLs.
• Stable Diffusion Prompt Extractor: read prompts and settings embedded in AI images.
• Japanese to Romaji, Hiragana & Furigana Converter: readings for Japanese text.
• Japan Diet Parliament Debates & Speeches Search: speeches from Japan's National Diet since 1947.