AI Readiness Checker — AEO Checker for Web Pages

AI Readiness Checker is an AEO checker that tests public web pages for answer engine optimization and returns, for each page, a 0–100 AI readiness score with a letter grade, robots.txt access for six AI crawlers, llms.txt and sitemap presence, JSON-LD types, content structure and trust signals, and a list of issues with severity, score impact and a fix. It costs USD 50 per 1,000 page audits on the FREE tier (USD 0.050001 per page), with a start fee of USD 0.006251.

It is built for SEO and GEO consultants, site owners and AI agents that need a repeatable, rule-based AEO check across a list of URLs. Every check is deterministic HTML and robots.txt analysis; no AI model is involved, so the same page gives the same result.

What does the AEO checker test on each page?

Each audited page becomes one dataset row with 39 fields.

Field	What it tells you
sourceUrl, finalUrl, statusCode, contentType, responseTimeMs	Requested URL, URL after redirects, HTTP status, content type and HTML response time
pageTitle, metaDescription, canonicalUrl, language, robotsMeta	Core page metadata
indexable	Whether page-level robots directives allow indexing
aiCrawlable	Whether page directives and robots.txt leave at least one checked AI crawler allowed
aiBotAccess	allowed, blocked or unspecified for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and Google-Extended
robotsTxtFound, sitemapFound, llmsTxtFound	Site files found at the origin
h1Count, headingCount, questionHeadingCount	Heading structure, including question-style headings
wordCount, paragraphCount, listCount, tableCount, hasMainContent	Content depth and structure
structuredDataTypes, hasOrganizationSchema, hasContentSchema	JSON-LD types and whether entity and content schema exist
hasAuthor, hasPublishedDate, trustPageLinks, internalLinkCount, externalLinkCount	Trust and citation signals
score, grade, scoreBreakdown	Total score, grade and the five category scores
passedChecks, failedChecks, issues	Check counts and the issue list (code, category, severity, message, recommendation, scoreImpact)
auditedAt	UTC timestamp

The run also writes an OUTPUT record (copied to SUMMARY) with the outcome, pages audited, failed requests, pages skipped by robots.txt, the average score and grade, the ten most costly issues across the run, and warnings.

Which AI crawlers does it check in robots.txt?

It reads /robots.txt once per site and reports root-path access for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and Google-Extended, falling back to wildcard rules when a crawler has no group of its own. A site with no robots.txt reports unspecified for every crawler. Blocking all six costs 10 points; blocking some costs 4.

How is the AI readiness score calculated?

Every page starts with 100 points split into access (25), machine readability (20), content (25), trust (15) and technical (15). Each failed check subtracts its scoreImpact from its category, for example a noindex page (−12), no JSON-LD (−8), thin content under 150 words (−6), a missing canonical (−4), no llms.txt (−2), or an HTTP error status. Grades are A (90+), B (80–89), C (70–79), D (60–69) and F (below 60). Issues are sorted by score impact, so the first one is the biggest win.

How do I run an AEO check on my site?

Add one or more URLs. With crawlLinks on, the Actor follows same-site links from each start page until it reaches maxPages.

{
  "startUrls": [{ "url": "https://www.example.com/" }, { "url": "https://www.example.com/pricing" }],
  "maxPages": 20,
  "crawlLinks": true,
  "respectRobotsTxt": true,
  "maxConcurrency": 3,
  "requestTimeoutSecs": 30
}

• startUrls (required): absolute HTTP(S) URLs. The default and prefill audit https://example.com/.
• maxPages: 1–50, default 5. This is the hard cap on page audits and charges.
• crawlLinks (default true): turn off to audit only the URLs you list.
• respectRobotsTxt (default true): skips discovered links disallowed for generic crawlers; start URLs are always fetched.
• maxConcurrency: 1–10 parallel requests, default 3. requestTimeoutSecs: 5–60, default 30.

A page that returns a 4xx status is not audited and not charged; it is listed under httpErrorPages in OUTPUT. URLs that redirect to a page already audited in the run are skipped without charge and counted in duplicateFinalUrls.

When should I use this AEO checker, and when not?

Use it to check a list of key pages, or the first pages of a site, against a fixed AI-readiness checklist and to track the score after fixes.

Do not use it for:

• A whole-site crawl with a site-level summary of the most common issues. Use AI Search Visibility Audit, which seeds from your sitemap and crawls up to 100 pages of one site.
• Accessibility, Open Graph and page-weight checks alongside AI readiness. Use AI SEO Audit.
• An AI model's judgment of how answerable your content is. Use GEO Content Auditor.
• Whether AI assistants actually mention your brand. Use AI Visibility Tracker.
• Classic on-page SEO and security headers. Use SEO Site Checkup.

Output example

Abridged row from the default run on 2026-10-26 (https://example.com/).

{
  "sourceUrl": "https://example.com/",
  "finalUrl": "https://example.com/",
  "statusCode": 200,
  "pageTitle": "Example Domain",
  "metaDescription": null,
  "indexable": true,
  "aiCrawlable": true,
  "robotsTxtFound": false,
  "sitemapFound": false,
  "llmsTxtFound": false,
  "aiBotAccess": { "gptBot": "unspecified", "oaiSearchBot": "unspecified", "chatGptUser": "unspecified", "claudeBot": "unspecified", "perplexityBot": "unspecified", "googleExtended": "unspecified" },
  "h1Count": 0,
  "wordCount": 25,
  "structuredDataTypes": [],
  "score": 45,
  "grade": "F",
  "scoreBreakdown": { "access": 19, "machineReadability": 6, "content": 7, "trust": 2, "technical": 11 },
  "passedChecks": 8,
  "failedChecks": 15,
  "issues": [
    { "code": "STRUCTURED_DATA_MISSING", "category": "machineReadability", "severity": "warning", "message": "No JSON-LD types were detected.", "recommendation": "Add valid schema.org JSON-LD that matches the visible page content.", "scoreImpact": 8 },
    { "code": "THIN_CONTENT", "category": "content", "severity": "warning", "message": "Only 25 readable words were detected.", "recommendation": "Add original, useful content that directly answers the page topic.", "scoreImpact": 6 }
  ]
}

Pricing: how much does an AEO check cost?

Pay per event; Apify platform usage is included. One event is charged for each page audit written to the dataset.

Event	FREE	BRONZE	SILVER	GOLD / PLATINUM / DIAMOND
Actor start (per started GB of allocated memory)	USD 0.006251	USD 0.006094	USD 0.005938	USD 0.005
Page readiness audit (primary)	USD 0.050001	USD 0.04875	USD 0.0475	USD 0.04

The default memory is 512 MB, so a normal run has one start event.

• Default input, 1 page (FREE): 0.006251 + 0.050001 = about USD 0.06.
• 50 pages (FREE): 0.006251 + 50 × 0.050001 = about USD 2.51.

Requests that fail after retries produce no row and no charge. maxPages caps the number of charged page audits.

How reliable is it?

The default run on example.com took about 5 seconds. Each request has a 5–60 second timeout and up to two retries; failed URLs are listed in the OUTPUT warnings, and the run fails with a clear message only when no page could be audited. The Actor reads server-delivered HTML, so content rendered only by JavaScript is not seen. It identifies itself with its own user agent and always uses its own direct connection; sites that block unknown bots may refuse it, and those URLs are listed as failed in OUTPUT and not charged.

Use it from an AI assistant

AI Readiness Checker works from Claude, ChatGPT, Cursor or any MCP client through Apify's MCP server. Add https://mcp.apify.com/?tools=muhammadafzal/ai-readiness-checker to your client.

Example prompt: "Run an AEO check on our homepage and pricing page, tell me which AI crawlers robots.txt blocks, and list the three issues with the biggest score impact."

Run it from the API

JavaScript (apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('muhammadafzal/ai-readiness-checker').call({
  startUrls: [{ url: 'https://www.example.com/' }],
  maxPages: 10,
  crawlLinks: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
const output = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('OUTPUT');
console.log(items, output?.value);

Python (apify-client 3.x):

from apify_client import ApifyClient

client = ApifyClient(token='YOUR_APIFY_TOKEN')
run = client.actor('muhammadafzal/ai-readiness-checker').call(run_input={
    'startUrls': [{'url': 'https://www.example.com/'}],
    'maxPages': 10,
})
items = client.dataset(run.default_dataset_id).list_items().items

FAQ

What is an AEO checker?

An AEO (answer engine optimization) checker tests whether AI answer engines can reach, parse and trust a page: crawler access, structured data, answer-friendly structure and trust signals. This Actor runs those checks and scores each page.

Does a high score mean AI assistants will cite my page?

No. The score covers on-page and robots.txt factors you control. Citations also depend on authority and the question asked; measure them with the AI Visibility Tracker.

Do I need an llms.txt file?

It is optional. A missing /llms.txt is reported as an info-level issue worth 2 points.

Why do all crawlers show "unspecified"?

The site has no robots.txt, or robots.txt has no rule for those crawlers and no wildcard rule for the root path.

Can it audit pages behind a login?

No. It audits public pages only and does not bypass authentication, paywalls or CAPTCHAs.

How many pages can one run check?

Up to 50 (maxPages). Split larger sites into several runs or use the AI Search Visibility Audit for up to 100 pages of one site.

Related Actors

• AI Search Visibility Audit: one-site crawl with sitemap seeding and a site summary.
• AI SEO Audit: crawlability, schema, accessibility, page weight and AI bot access.
• GEO Content Auditor: AI-scored answerability and quick wins per page.
• AI Visibility Tracker: brand mentions, rank and share of voice in AI answers.
• AEO Tracker: AI answer visibility monitoring over time.

Responsible use

Check only public pages you own or are allowed to analyze, keep request rates reasonable and leave respectRobotsTxt on unless you have permission. The Actor does not log in, bypass access controls or solve CAPTCHAs.

Last verified: 2026-10-27