# SEO Audit & AI Search Readiness (`appreciative_duct/seo-audit-ai-search-readiness`) Actor

SEO audit for any page or website, plus AI search readiness: titles, canonicals, schema, llms.txt, and whether GPTBot, ClaudeBot, and PerplexityBot can read the site.

- **URL**: https://apify.com/appreciative_duct/seo-audit-ai-search-readiness.md
- **Developed by:** [Ross AI](https://apify.com/appreciative_duct) (community)
- **Categories:** SEO tools, Marketing, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit & AI Search Readiness

This SEO audit scores a page from 0 to 100 and lists the fixes in priority order. It checks the on-page basics search consoles already care about, then adds the AI-search checks that decide whether ChatGPT, Claude, Perplexity, and Google's AI features can read the page.

The Actor uses plain HTTP and an HTML parser. It does not launch a browser. What you see in the result is the HTML a crawler receives before JavaScript runs. If the page is an empty application shell, the audit says so.

### Who the SEO audit is for

- SEO and content teams reviewing a template before a release.
- Agencies sampling a client site from its sitemap.
- Site owners who want one report for classic SEO and AI-search readiness, instead of a separate GEO tool.
- Developers checking that a redesign still ships a title, a canonical, and indexable HTML.

Pair it with the [Sitemap URL Extractor](https://apify.com/appreciative_duct/sitemap-url-extractor) when you need the full URL list first. This Actor can also read a sitemap itself and stop after the page limit you set.

### What the score includes

Each check has a weight. The score is the sum, rounded to 0–100.

| Area | What is checked |
| --- | --- |
| Indexability | Final HTTP status, noindex in meta robots or `X-Robots-Tag`, response time, redirect hops |
| On-page | Title length, meta description length, a single H1 |
| Technical | Canonical, `html lang`, hreflang links, JSON-LD |
| Content | Word count in the initial HTML, JavaScript-only shells, image alt coverage, internal links, heading order |
| Social | Open Graph title, description, and image; Twitter card |
| AI search | `robots.txt` rules for GPTBot, ClaudeBot, PerplexityBot, and Google-Extended; `/llms.txt`; FAQ, HowTo, Article, or Organization schema; an answer in the opening paragraph; author and date signals |

Grades: 90+ excellent, 75+ good, 50+ fair, below 50 poor.

A missing hreflang set is not a penalty. Many sites are one language. Invalid hreflang links are.

`robots.txt` is read once per site. A bot with no rule of its own inherits the `User-agent: *` group. No rule means the bot is allowed. The check uses the path of the page, so a bot blocked only on `/account` is still allowed on `/blog`.

### Input

Audit three specific pages:

```json
{
  "startUrls": [
    { "url": "https://www.python.org/" },
    { "url": "https://www.python.org/downloads/" },
    { "url": "https://www.python.org/about/" }
  ],
  "discoverFromSitemap": false
}
```

Or sample a site from its sitemap:

```json
{
  "startUrls": [{ "url": "https://apify.com" }],
  "maxPages": 10,
  "discoverFromSitemap": true,
  "excludeUrlPatterns": ["/blog/tag/"]
}
```

| Field | Default | What it does |
| --- | --- | --- |
| `startUrls` | required | Page URLs, a homepage, or a sitemap URL. A bare domain is treated as HTTPS. |
| `maxPages` | 25 | Stop after this many pages. The console form starts at 5. |
| `discoverFromSitemap` | true | Expand homepages and sitemap URLs through `robots.txt` and common sitemap paths. Deeper URLs are audited as typed. |
| `sameDomainOnly` | true | Keep discovered URLs on the same hostname. `www` is ignored. |
| `includeUrlPatterns` / `excludeUrlPatterns` | empty | Case-insensitive regular expressions for discovered URLs. Typed URLs are always audited. |
| `maxConcurrency` | 5 | Parallel page fetches. |
| `maxRetries` | 2 | Retries for timeouts and HTTP 429 or 5xx. |
| `requestTimeoutSecs` | 20 | Per-request timeout. |
| `maxRedirects` | 5 | Redirect hops to follow. Each hop is recorded. |

### Output

One dataset item per page. The overview table shows the score. The AI search view shows crawler access.

```json
{
  "url": "https://www.python.org/",
  "finalUrl": "https://www.python.org/",
  "statusCode": 200,
  "redirectCount": 0,
  "responseTimeMs": 380,
  "score": 74,
  "grade": "fair",
  "title": "Welcome to Python.org",
  "titleLength": 22,
  "metaDescriptionLength": 0,
  "h1": "Python",
  "h1Count": 1,
  "canonical": "https://www.python.org/",
  "noindex": false,
  "htmlLang": "en",
  "wordCount": 420,
  "imageCount": 6,
  "imagesWithAlt": 5,
  "imageAltCoverage": 0.83,
  "internalLinkCount": 40,
  "externalLinkCount": 3,
  "jsonLdTypes": ["Organization"],
  "likelyRequiresJavaScript": false,
  "answerFirst": true,
  "hasAuthor": false,
  "hasDate": false,
  "gptBot": "allowed",
  "claudeBot": "allowed",
  "perplexityBot": "allowed",
  "googleExtended": "allowed",
  "llmsTxt": false,
  "topFix": "Add a meta description of about 70–160 characters that answers what the page is about.",
  "fixCount": 6,
  "fixes": [
    {
      "priority": "high",
      "id": "meta-description",
      "message": "Meta description is missing.",
      "fix": "Add a meta description of about 70–160 characters that answers what the page is about."
    }
  ]
}
```

`checks` contains every scored rule, including the ones that passed. `SUMMARY` in the key-value store has the average score, grade counts, and the fixes that appeared most often.

A page that cannot be fetched is still a row, with score 0 and the error in `error`, so a blocked URL is visible instead of disappearing.

### API

```bash
curl "https://api.apify.com/v2/acts/appreciative_duct~seo-audit-ai-search-readiness/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://apify.com"}],"maxPages":5}'
```

### Pricing

The Actor is free. You pay only Apify platform usage. It defaults to 256 MB and does not use a browser or a proxy.

The code charges one `page-audited` event per saved page when pay-per-event pricing is turned on. It is not turned on yet.

### FAQ

#### What does this SEO audit actually fetch?

The URL you asked for, plus, once per website, `/robots.txt` and `/llms.txt`. Homepages and sitemap URLs also download sitemap files until `maxPages` page URLs are collected. Redirects are followed and listed. Response bodies larger than 2 MB are refused.

#### Does the audit run JavaScript?

No. A page whose text appears only after hydration gets a low word count and a "needs JavaScript" flag. That is the useful result: AI crawlers that do not render JavaScript cannot quote that page either.

#### How are GPTBot, ClaudeBot, PerplexityBot, and Google-Extended judged?

The Actor parses `robots.txt` groups. A group named for that bot overrides `User-agent: *`. The longest matching path rule wins, and `Allow` beats `Disallow` when the lengths are equal. The result is `allowed` or `disallowed` for the path that was audited.

#### What counts as llms.txt?

`https://example.com/llms.txt` must return HTTP 200, must not be an HTML error page, and must contain at least a short text body. A redirect to the homepage does not count.

#### Why is my score not 100?

Open `fixes`. High-priority items are missing titles, missing descriptions, noindex, or a non-200 status. AI-search points are lost when a named crawler is blocked, `/llms.txt` is missing, the page does not lead with an answer, or there is no author, date, or FAQ/HowTo/Article/Organization schema.

#### Can I audit a whole site?

Set `maxPages` as high as 500. For a complete URL inventory, run the [Sitemap URL Extractor](https://apify.com/appreciative_duct/sitemap-url-extractor) first, then pass the URLs you care about. Five hundred HTML fetches is a sample, not a replacement for a full crawl.

#### A page failed. Did the run fail?

No. The row stores the error and a score of 0. The run fails only when there is nothing to audit, for example an empty `startUrls` list or a rejected private address.

### Notes

Private, localhost, and link-local addresses are refused. The Actor does not solve CAPTCHAs or use a proxy. Use it on sites you are allowed to request.

# Changelog

This Actor's version history is a separate document: https://apify.com/appreciative_duct/seo-audit-ai-search-readiness/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

Pages to audit, a website homepage, or a sitemap URL. A homepage is expanded from its sitemap up to the page limit. A deeper URL is audited on its own.

## `maxPages` (type: `integer`):

How many pages to audit in this run. The homepage counts toward the limit when you pass a website.

## `discoverFromSitemap` (type: `boolean`):

When a start URL is a homepage or a sitemap file, collect more pages from the sitemap until the page limit is reached.

## `sameDomainOnly` (type: `boolean`):

Drop sitemap URLs on a different hostname. www and the bare domain are treated as the same site.

## `includeUrlPatterns` (type: `array`):

Optional case-insensitive JavaScript regular expressions. Applied to pages discovered from a sitemap. URLs you typed are always audited.

## `excludeUrlPatterns` (type: `array`):

Drop discovered pages that match any of these regular expressions.

## `maxConcurrency` (type: `integer`):

How many pages to fetch at the same time.

## `maxRetries` (type: `integer`):

Retries for timeouts, connection errors, HTTP 429, and HTTP 5xx.

## `requestTimeoutSecs` (type: `integer`):

Time limit for one page, robots.txt, llms.txt, or sitemap request.

## `maxRedirects` (type: `integer`):

Redirect hops to follow before the page is reported as a redirect error.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "maxPages": 5,
  "discoverFromSitemap": true,
  "sameDomainOnly": true,
  "includeUrlPatterns": [],
  "excludeUrlPatterns": [],
  "maxConcurrency": 5,
  "maxRetries": 2,
  "requestTimeoutSecs": 20,
  "maxRedirects": 5
}
```

# Actor output Schema

## `pages` (type: `string`):

One dataset item per page. Includes the SEO score, on-page fields, AI crawler access, and the fix list.

## `summary` (type: `string`):

Average score, grade counts, and the fixes that showed up most often.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        }
    ],
    "maxPages": 5,
    "includeUrlPatterns": [],
    "excludeUrlPatterns": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("appreciative_duct/seo-audit-ai-search-readiness").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://apify.com" }],
    "maxPages": 5,
    "includeUrlPatterns": [],
    "excludeUrlPatterns": [],
}

# Run the Actor and wait for it to finish
run = client.actor("appreciative_duct/seo-audit-ai-search-readiness").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "maxPages": 5,
  "includeUrlPatterns": [],
  "excludeUrlPatterns": []
}' |
apify call appreciative_duct/seo-audit-ai-search-readiness --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,appreciative_duct/seo-audit-ai-search-readiness"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/E79sPm8In4XOF9dto/builds/TcWe9fpUXETkvsjVj/openapi.json