# On Page SEO Audit Scraper (`automation-lab/multi-page-on-page-seo-audit`) Actor

Crawl public websites and export page-level on-page SEO metadata, indexability, links, content signals, and issue flags.

- **URL**: https://apify.com/automation-lab/multi-page-on-page-seo-audit.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.40 / 1,000 item extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## On Page SEO Audit Scraper

Run a bounded **on page SEO audit** across one or more public websites and receive one structured dataset row per page. The Actor follows same-site links and reports HTTP status, titles, descriptions, headings, canonical URLs, robots directives, indexability, links, content signals, structured data types, image alt coverage, and deterministic issue flags.

Use the output for a one-time website review, recurring technical SEO audits, migration checks, spreadsheet exports, or your own reporting pipeline. This is an HTTP-first crawler: it audits anonymous server-rendered HTML without launching a browser or silently activating a paid proxy.

### What does this on page SEO crawler do?

For every supplied start URL, the Actor:

1. validates that the URL resolves to a public network address;
2. requests the page with bounded redirects and retries;
3. extracts page-level on page SEO fields from HTML and response headers;
4. discovers normalized same-site links;
5. stops at your page, depth, link, and concurrency limits;
6. compares collected titles and descriptions for duplicates;
7. saves one typed record for every attempted page.

HTTP errors and non-HTML URLs are still represented as records. A terminal request failure gets a `REQUEST_FAILED` row instead of disappearing from an audit.

### Who is it for?

- **SEO consultants** auditing small and medium public websites.
- **In-house SEO teams** checking releases, migrations, and recurring changes.
- **Developers** validating metadata and indexability in CI or scheduled jobs.
- **Agencies** collecting the same normalized fields across multiple client sites.
- **Data teams** loading page-level technical SEO records into a warehouse or dashboard.

The Actor is designed for page-source signals. Choose [Lighthouse Website Audit](https://apify.com/automation-lab/website-lighthouse-seo-audit) when you need Lighthouse performance, accessibility, best-practice, and SEO scores instead.

### Why use this Actor?

- **Multi-page output:** one reusable row per page rather than one site-level summary.
- **Deterministic flags:** stable issue codes are easy to filter, compare, and alert on.
- **Bounded crawling:** explicit page, depth, per-page link, timeout, and concurrency controls.
- **Multi-site runs:** start URLs may belong to unrelated public websites.
- **Safe URL handling:** local, private, reserved, credential-bearing, and unsafe redirect targets are rejected.
- **Cost-aware HTTP route:** no browser or automatic residential proxy fallback.
- **Integration-ready schema:** normalized URLs, nullable failure fields, arrays, counts, and timestamps.

### What on page SEO data is extracted?

| Field | Meaning |
| --- | --- |
| `url`, `finalUrl`, `startUrl` | Requested URL, redirect destination, and crawl seed |
| `depth` | Link distance from the supplied start URL |
| `statusCode`, `contentType` | Final HTTP response information |
| `responseTimeMs` | Request and redirect elapsed time |
| `title`, `titleLength` | Normalized HTML title and character count |
| `metaDescription`, `metaDescriptionLength` | Meta description and character count |
| `h1`, `h2` | All non-empty H1 and H2 texts |
| `canonicalUrl` | Resolved valid canonical URL, or `null` |
| `robots`, `indexable` | Robots directives and calculated indexability |
| `language` | HTML `lang` value |
| `wordCount`, `textLength` | Approximate visible-body content signals |
| `internalLinkCount`, `externalLinkCount` | Unique link counts |
| `internalLinks`, `externalLinks` | Up to 100 normalized URLs of each kind |
| `imageCount`, `imagesMissingAlt` | Image and missing-alt counts |
| `structuredDataTypes` | Unique JSON-LD `@type` values |
| `issueFlags` | Sorted deterministic page issue codes |
| `error` | Concise terminal request error, otherwise `null` |
| `auditedAt` | ISO 8601 audit timestamp |

### Which SEO issue flags are reported?

A record may contain any of these codes:

- `HTTP_ERROR`
- `NON_HTML`
- `REQUEST_FAILED`
- `MISSING_TITLE`
- `TITLE_TOO_SHORT`
- `TITLE_TOO_LONG`
- `DUPLICATE_TITLE`
- `MISSING_META_DESCRIPTION`
- `META_DESCRIPTION_TOO_SHORT`
- `META_DESCRIPTION_TOO_LONG`
- `DUPLICATE_META_DESCRIPTION`
- `MISSING_H1`
- `MULTIPLE_H1`
- `MISSING_CANONICAL`
- `INVALID_CANONICAL`
- `NOINDEX`
- `MISSING_LANGUAGE`
- `LOW_WORD_COUNT`
- `IMAGES_MISSING_ALT`

Threshold flags are diagnostic signals, not universal ranking rules. Review them in the context of page purpose, language, templates, and search strategy.

### How to run your first website audit

1. Open the Actor in Apify Console.
2. Add one or more public pages to **Start URLs**.
3. Set **Maximum pages** to a small value such as `10` for the first run.
4. Choose a link depth; `2` is a useful bounded preview.
5. Keep concurrency low for a small or rate-limited website.
6. Click **Start**.
7. Open the default dataset and select the **SEO audit** view.
8. Export rows as JSON, CSV, Excel, XML, or another Apify-supported format.

A practical starter input is:

```json
{
  "startUrls": [{ "url": "https://www.w3.org/" }],
  "maxPages": 10,
  "maxDepth": 2,
  "maxConcurrency": 2,
  "includeSubdomains": false,
  "followQueryParameters": false
}
```

### Input parameters

| Input | Default | Limits | Description |
| --- | ---: | ---: | --- |
| `startUrls` | required | 1 or more | Public HTTP(S) crawl seeds |
| `maxPages` | `10` | 1–10,000 | Total output page records across all sites |
| `maxDepth` | `3` | 0–10 | Same-site link levels followed from each seed |
| `maxConcurrency` | `3` | 1–20 | Page requests processed at once |
| `includeSubdomains` | `false` | boolean | Whether discovered subdomain links are in scope |
| `followQueryParameters` | `false` | boolean | Whether query variants count as separate pages |
| `maxLinksPerPage` | `20` | 1–5,000 | Discovery links accepted from each page |
| `requestTimeoutSecs` | `30` | 5–120 | Per-request timeout before bounded retries |

When query following is disabled, query strings are removed during URL normalization. When enabled, common tracking parameters such as `utm_*`, `fbclid`, and `gclid` are still removed.

### Output example

This shortened record reflects real current output from a W3C page audit:

```json
{
  "url": "https://www.w3.org/standards/",
  "finalUrl": "https://www.w3.org/standards/",
  "startUrl": "https://www.w3.org/",
  "depth": 1,
  "statusCode": 200,
  "contentType": "text/html; charset=utf-8",
  "title": "Web Standards | W3C",
  "titleLength": 19,
  "metaDescriptionLength": 61,
  "h1": ["Web standards"],
  "canonicalUrl": null,
  "indexable": true,
  "wordCount": 738,
  "internalLinkCount": 31,
  "externalLinkCount": 4,
  "issueFlags": [
    "IMAGES_MISSING_ALT",
    "META_DESCRIPTION_TOO_SHORT",
    "MISSING_CANONICAL",
    "TITLE_TOO_SHORT"
  ],
  "error": null,
  "auditedAt": "2026-08-24T00:00:00.000Z"
}
```

Fields that cannot be obtained are `null` rather than invented. Arrays are empty when no values are found.

### How much does it cost to audit website pages?

Pay-per-event pricing consists of a **$0.005 start fee** and one page-audit event for every dataset row. The BRONZE page price is **$0.004 per page audit**; higher plan tiers receive lower page prices.

Examples at BRONZE pricing:

| Useful page records | Estimated charge |
| ---: | ---: |
| 1 | $0.0090 |
| 10 | $0.0450 |
| 25 | $0.1050 |
| 100 | $0.4050 |

Failed-request and HTTP-error rows are page audit records because they preserve actionable coverage evidence. No separate charge exists for links, headings, flags, or structured-data fields included in a row.

Apify compute usage is covered by pay-per-event pricing for customers. The exact active price is always displayed in Console before a run.

### Recurring technical SEO audit workflow

For recurring monitoring:

1. save a Task with stable start URLs and crawl limits;
2. schedule it daily, weekly, or after deployments;
3. export each run dataset or send it through a webhook;
4. join records by `url`;
5. compare `statusCode`, `indexable`, metadata fields, and `issueFlags`;
6. alert only on changes important to your site.

The Actor emits current snapshots. It does not maintain history or send alerts by itself; Apify schedules, webhooks, datasets, and your downstream system provide those workflow layers.

### Spreadsheet and data pipeline integrations

You can:

- export the dataset directly to CSV or Excel;
- connect Apify with Google Sheets through supported integrations;
- trigger a Make or Zapier workflow after a run;
- retrieve JSON through the dataset API;
- load normalized rows into BigQuery, Snowflake, PostgreSQL, or a lake;
- filter `issueFlags` in a dashboard;
- compare successive scheduled datasets in your own model.

Keep `url` as the page identity and preserve `startUrl` when one run contains several websites.

### Run with the Apify API

Replace `APIFY_TOKEN` with your token.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~multi-page-on-page-seo-audit/runs?token=APIFY_TOKEN&waitForFinish=300" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://www.w3.org/"}],"maxPages":10,"maxDepth":2}'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/multi-page-on-page-seo-audit').call({
  startUrls: [{ url: 'https://www.w3.org/' }],
  maxPages: 10,
  maxDepth: 2,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient("APIFY_TOKEN")
run = client.actor("automation-lab/multi-page-on-page-seo-audit").call(run_input={
    "startUrls": [{"url": "https://www.w3.org/"}],
    "maxPages": 10,
    "maxDepth": 2,
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

### Use with MCP and AI agents

Add this Actor as a tool in Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/multi-page-on-page-seo-audit"
```

**Claude Desktop setup:** add the following remote MCP server configuration.

**Cursor setup:** open MCP settings and add the same remote server entry.

**VS Code setup:** add the server entry to your MCP configuration file.

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/multi-page-on-page-seo-audit"
    }
  }
}
```

Example prompts:

- “Audit up to 20 pages from `https://www.w3.org/` and summarize noindex or missing-title records.”
- “Run the multi-site technical SEO audit and return pages with HTTP errors.”
- “Export URLs whose canonical is missing and word count is below 200.”

### Legality and responsible use

Only crawl public pages you are authorized to access. Follow applicable terms, policies, copyright rules, privacy law, and website operator guidance. Use conservative limits and concurrency. Do not use the Actor to probe private networks: private, loopback, link-local, reserved, credential-bearing, and unsafe redirect destinations are blocked.

This Actor does not log in, solve CAPTCHAs, bypass access controls, or automatically switch to paid residential proxies. A public website may still rate-limit or block automated requests.

### Limitations

- Client-rendered metadata that is absent from server HTML is not visible.
- The crawler follows HTML anchors; it does not ingest XML sitemaps in this release.
- Query URLs are merged by default to reduce crawl traps.
- Link counts describe links on the audited page; linked destinations are not labeled broken unless their own URL is crawled and returns an error row.
- `indexable` reflects response status, HTML content type, and noindex directives. It is not a search-engine indexing guarantee.
- Word count is an approximate normalized visible-body count.
- Duplicate flags compare only rows in the current run.
- A page may change between recurring runs.
- Authenticated, browser-only, CAPTCHA-protected, and private pages are outside scope.

### Tips for reliable audits

- Start with 10–25 pages and inspect discovered URL patterns.
- Leave query following off unless query variants are genuine pages.
- Lower concurrency when a site returns 429 or intermittent 5xx responses.
- Supply important section pages as additional start URLs when they are not linked near the root.
- Increase depth gradually; deep calendars and faceted navigation can expose many URLs.
- Keep the same inputs for comparable recurring snapshots.
- Treat content-length thresholds as review cues rather than automatic SEO verdicts.

### Troubleshooting

#### Why did the run return fewer pages than `maxPages`?

The same-site crawl may have exhausted unique eligible links, reached `maxDepth`, skipped file assets, or merged query variants. `maxPages` is a ceiling, not a promised count.

#### Why is a page a `REQUEST_FAILED` row?

The page did not return a final response after bounded retries. Check `error`, reduce concurrency, verify the URL in a normal unauthenticated request, and try a smaller audit. The Actor retries transient network, 429, and 5xx failures, not stable parser or input errors.

#### Why did the Actor reject a URL?

Only anonymous public HTTP(S) destinations are supported. Local/private addresses, single-label hosts, credential-bearing URLs, and redirects to restricted networks fail closed.

#### Why is JavaScript-rendered content missing?

The Actor intentionally parses HTTP HTML without a browser. Use a public server-rendered or alternate URL, or choose a browser-based audit product when rendered DOM is essential.

### Related Automation Lab Actors

- [Lighthouse Website Audit](https://apify.com/automation-lab/website-lighthouse-seo-audit) — Lighthouse performance, accessibility, best-practice, and SEO audit scores.
- [Broken Link Checker](https://apify.com/automation-lab/broken-link-checker) — destination-focused broken-link checks.
- [Website HTML & Text Change Monitor](https://apify.com/automation-lab/website-html-text-change-monitor) — versioned HTML and normalized text changes.
- [Multi-Site Article Content Extractor](https://apify.com/automation-lab/multi-site-article-content-extractor) — article-focused text and metadata extraction from supplied URLs.

### FAQ

#### Does it crawl more than one website?

Yes. Add unrelated public start URLs and set one total `maxPages` budget. Every row preserves its originating `startUrl`.

#### Does it execute JavaScript?

No. The HTTP-first design is faster and cheaper for server-rendered HTML but cannot see metadata created only after browser execution.

#### Does it use a proxy?

No automatic proxy mode is exposed. Direct anonymous HTTP is used for predictable cost and behavior.

#### Are errors charged?

Every saved page audit row uses the page-audit event, including HTTP-error and terminal request-failure rows because they are actionable audit results. Invalid run input fails before page processing.

#### Can I schedule it?

Yes. Save the input as an Apify Task and attach a schedule. Store each run dataset or forward results through an integration for historical comparison.

#### Can the output prove that Google indexed a page?

No. `indexable` is a deterministic technical signal from the fetched response and directives, not evidence from a search-engine index.

#### Will it publish or modify my website?

No. It performs read-only HTTP requests and writes results only to the Actor run dataset.

# Actor input Schema

## `startUrls` (type: `array`):

Public HTTP(S) pages where each bounded same-site crawl begins. You can audit multiple websites in one run.

## `maxPages` (type: `integer`):

Maximum total page audit records saved across all supplied websites.

## `maxDepth` (type: `integer`):

Number of same-site link levels followed from each start URL. Zero audits only supplied URLs.

## `maxConcurrency` (type: `integer`):

Maximum page requests processed at once. Keep this low for small or rate-limited websites.

## `includeSubdomains` (type: `boolean`):

Follow links on subdomains of each supplied website in addition to its main hostname.

## `followQueryParameters` (type: `boolean`):

Treat URLs with different query parameters as separate pages. Disabled by default to avoid faceted-navigation crawl traps.

## `maxLinksPerPage` (type: `integer`):

Maximum same-site links accepted from one page for crawl scheduling.

## `requestTimeoutSecs` (type: `integer`):

Maximum time allowed for each page request before bounded retries are attempted.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.w3.org/"
    },
    {
      "url": "https://www.apache.org/"
    }
  ],
  "maxPages": 10,
  "maxDepth": 3,
  "maxConcurrency": 3,
  "includeSubdomains": false,
  "followQueryParameters": false,
  "maxLinksPerPage": 20,
  "requestTimeoutSecs": 30
}
```

# Actor output Schema

## `overview` (type: `string`):

Open the normalized page-level SEO audit rows in the overview dataset view.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.w3.org/"
        },
        {
            "url": "https://www.apache.org/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/multi-page-on-page-seo-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://www.w3.org/" },
        { "url": "https://www.apache.org/" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/multi-page-on-page-seo-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.w3.org/"
    },
    {
      "url": "https://www.apache.org/"
    }
  ]
}' |
apify call automation-lab/multi-page-on-page-seo-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/multi-page-on-page-seo-audit"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ZR85IYMCtSBeiWSpG/builds/wRmGnkZhJ1Nj6Hm6X/openapi.json