SEO Audit & AI Search Readiness

This SEO audit scores a page from 0 to 100 and lists the fixes in priority order. It checks the on-page basics search consoles already care about, then adds the AI-search checks that decide whether ChatGPT, Claude, Perplexity, and Google's AI features can read the page.

The Actor uses plain HTTP and an HTML parser. It does not launch a browser. What you see in the result is the HTML a crawler receives before JavaScript runs. If the page is an empty application shell, the audit says so.

Who the SEO audit is for

• SEO and content teams reviewing a template before a release.
• Agencies sampling a client site from its sitemap.
• Site owners who want one report for classic SEO and AI-search readiness, instead of a separate GEO tool.
• Developers checking that a redesign still ships a title, a canonical, and indexable HTML.

Pair it with the Sitemap URL Extractor when you need the full URL list first. This Actor can also read a sitemap itself and stop after the page limit you set.

What the score includes

Each check has a weight. The score is the sum, rounded to 0–100.

Area	What is checked
Indexability	Final HTTP status, noindex in meta robots or X-Robots-Tag, response time, redirect hops
On-page	Title length, meta description length, a single H1
Technical	Canonical, html lang, hreflang links, JSON-LD
Content	Word count in the initial HTML, JavaScript-only shells, image alt coverage, internal links, heading order
Social	Open Graph title, description, and image; Twitter card
AI search	robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, and Google-Extended; /llms.txt; FAQ, HowTo, Article, or Organization schema; an answer in the opening paragraph; author and date signals

Grades: 90+ excellent, 75+ good, 50+ fair, below 50 poor.

A missing hreflang set is not a penalty. Many sites are one language. Invalid hreflang links are.

robots.txt is read once per site. A bot with no rule of its own inherits the User-agent: * group. No rule means the bot is allowed. The check uses the path of the page, so a bot blocked only on /account is still allowed on /blog.

Input

Audit three specific pages:

{
  "startUrls": [
    { "url": "https://www.python.org/" },
    { "url": "https://www.python.org/downloads/" },
    { "url": "https://www.python.org/about/" }
  ],
  "discoverFromSitemap": false
}

Or sample a site from its sitemap:

{
  "startUrls": [{ "url": "https://apify.com" }],
  "maxPages": 10,
  "discoverFromSitemap": true,
  "excludeUrlPatterns": ["/blog/tag/"]
}

Field	Default	What it does
startUrls	required	Page URLs, a homepage, or a sitemap URL. A bare domain is treated as HTTPS.
maxPages	25	Stop after this many pages. The console form starts at 5.
discoverFromSitemap	true	Expand homepages and sitemap URLs through robots.txt and common sitemap paths. Deeper URLs are audited as typed.
sameDomainOnly	true	Keep discovered URLs on the same hostname. www is ignored.
includeUrlPatterns / excludeUrlPatterns	empty	Case-insensitive regular expressions for discovered URLs. Typed URLs are always audited.
maxConcurrency	5	Parallel page fetches.
maxRetries	2	Retries for timeouts and HTTP 429 or 5xx.
requestTimeoutSecs	20	Per-request timeout.
maxRedirects	5	Redirect hops to follow. Each hop is recorded.

Output

One dataset item per page. The overview table shows the score. The AI search view shows crawler access.

{
  "url": "https://www.python.org/",
  "finalUrl": "https://www.python.org/",
  "statusCode": 200,
  "redirectCount": 0,
  "responseTimeMs": 380,
  "score": 74,
  "grade": "fair",
  "title": "Welcome to Python.org",
  "titleLength": 22,
  "metaDescriptionLength": 0,
  "h1": "Python",
  "h1Count": 1,
  "canonical": "https://www.python.org/",
  "noindex": false,
  "htmlLang": "en",
  "wordCount": 420,
  "imageCount": 6,
  "imagesWithAlt": 5,
  "imageAltCoverage": 0.83,
  "internalLinkCount": 40,
  "externalLinkCount": 3,
  "jsonLdTypes": ["Organization"],
  "likelyRequiresJavaScript": false,
  "answerFirst": true,
  "hasAuthor": false,
  "hasDate": false,
  "gptBot": "allowed",
  "claudeBot": "allowed",
  "perplexityBot": "allowed",
  "googleExtended": "allowed",
  "llmsTxt": false,
  "topFix": "Add a meta description of about 70–160 characters that answers what the page is about.",
  "fixCount": 6,
  "fixes": [
    {
      "priority": "high",
      "id": "meta-description",
      "message": "Meta description is missing.",
      "fix": "Add a meta description of about 70–160 characters that answers what the page is about."
    }
  ]
}

checks contains every scored rule, including the ones that passed. SUMMARY in the key-value store has the average score, grade counts, and the fixes that appeared most often.

A page that cannot be fetched is still a row, with score 0 and the error in error, so a blocked URL is visible instead of disappearing.

API

curl "https://api.apify.com/v2/acts/appreciative_duct~seo-audit-ai-search-readiness/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://apify.com"}],"maxPages":5}'

Pricing

The Actor is free. You pay only Apify platform usage. It defaults to 256 MB and does not use a browser or a proxy.

The code charges one page-audited event per saved page when pay-per-event pricing is turned on. It is not turned on yet.

FAQ

What does this SEO audit actually fetch?

The URL you asked for, plus, once per website, /robots.txt and /llms.txt. Homepages and sitemap URLs also download sitemap files until maxPages page URLs are collected. Redirects are followed and listed. Response bodies larger than 2 MB are refused.

Does the audit run JavaScript?

No. A page whose text appears only after hydration gets a low word count and a "needs JavaScript" flag. That is the useful result: AI crawlers that do not render JavaScript cannot quote that page either.

How are GPTBot, ClaudeBot, PerplexityBot, and Google-Extended judged?

The Actor parses robots.txt groups. A group named for that bot overrides User-agent: *. The longest matching path rule wins, and Allow beats Disallow when the lengths are equal. The result is allowed or disallowed for the path that was audited.

What counts as llms.txt?

https://example.com/llms.txt must return HTTP 200, must not be an HTML error page, and must contain at least a short text body. A redirect to the homepage does not count.

Why is my score not 100?

Open fixes. High-priority items are missing titles, missing descriptions, noindex, or a non-200 status. AI-search points are lost when a named crawler is blocked, /llms.txt is missing, the page does not lead with an answer, or there is no author, date, or FAQ/HowTo/Article/Organization schema.

Can I audit a whole site?

Set maxPages as high as 500. For a complete URL inventory, run the Sitemap URL Extractor first, then pass the URLs you care about. Five hundred HTML fetches is a sample, not a replacement for a full crawl.

A page failed. Did the run fail?

No. The row stores the error and a score of 0. The run fails only when there is nothing to audit, for example an empty startUrls list or a rejected private address.

Notes

Private, localhost, and link-local addresses are refused. The Actor does not solve CAPTCHAs or use a proxy. Use it on sites you are allowed to request.