# SEO Site Audit - Crawl & Fix List for Your Website (`datagleaner/seo-site-audit`) Actor

SEO audit for your site: crawls the sitemap and links, checks status, redirects, titles, meta, H1, canonicals, noindex, hreflang, alt text, broken links, speed and schema. Score per page plus a ranked issues table. Honours robots.txt. $5 per 1,000 pages.

- **URL**: https://apify.com/datagleaner/seo-site-audit.md
- **Developed by:** [Data Gleaner](https://apify.com/datagleaner) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Site Audit - crawl your site, get a ranked fix list

Technical SEO audit for a website you own, manage or have been asked to review. Give it a domain: it reads `robots.txt`, starts from the sitemap (or the page you name), follows same-site links up to your page limit, and returns **one row per page with a 0-100 score and its issues**, plus a **site summary row with the issues ranked by severity**. Plain HTTP, no browser, no proxy.

### What it checks, per page

| Area | Checks |
|---|---|
| Status | HTTP status, redirect chain (every hop), redirect loops, unreachable pages |
| Titles and meta | Title and meta description: missing, too short, too long, duplicated across pages |
| Headings | H1 missing or more than one |
| Indexing | Canonical (missing or pointing elsewhere), noindex (meta tag or X-Robots-Tag), hreflang |
| Content | Word count (thin pages), images without alt text, structured-data types (JSON-LD, microdata) |
| Links | Internal and external link counts, broken internal links (targets that answered 4xx/5xx in the same crawl) |
| Performance | Response time (ms), HTML page weight (bytes) |
| Mobile | Viewport meta tag |
| Security | Mixed content: `http://` images, scripts, stylesheets and iframes on https pages (`mixed_content`, with `mixedContentCount` per page) |

Severity scale: critical (a page that errors or cannot be reached), high (missing title or H1, broken internal links), medium (redirect chains, noindex, duplicates, missing descriptions, missing alt text, slow or heavy pages), low (length and housekeeping issues). The page score is 100 minus 40 per critical, 15 per high, 7 per medium and 2 per low issue.

### How it behaves on your site

- User agent: `KeencrestBot/1.0 (+https://keencrest-guides.web.app/)`. It is always honest about who it is.
- It follows `robots.txt` for every URL it requests, including a `Crawl-delay`, and reports how many URLs it skipped (`robotsBlocked` on the summary row). It cannot be switched off.
- Default pace is one request about every 0.5 seconds, 3 at a time. On HTTP 429 or 503 it waits (honouring `Retry-After`) and backs off. If the site keeps throttling, or serves a bot-challenge page (Cloudflare "Just a moment", CAPTCHA), it stops and says so in the summary. It never rotates IPs or works around a block.
- Only the starting host is crawled (`www.` and the bare domain are one crawl scope: a page is audited once whichever form links to it, and links to the other form are not reported as outside links). When one form redirects to the other, pages whose internal links still use the redirecting form get the `links_to_redirecting_host` issue (medium): point links and the sitemap at the final host. If your start URL itself redirects to another origin, the summary row says where in `startUrlRedirectsTo`. Links with query strings are not followed; URLs listed in the sitemap are audited as given.

### Optional link check (off by default)

Set `checkLinks` to `true` and, after the crawl, every link on your pages that points to **another site**, or to **your own URLs the crawl did not fetch** (PDFs, images, pages past the page limit), is requested once and reported as `ok`, `broken` or `unverified`:

- It sends `HEAD` first and falls back to `GET` (without downloading the body) when HEAD fails or answers an error, so servers that refuse HEAD are not flagged by mistake.
- Each target host's `robots.txt` is read first. A URL it disallows is **not requested**, is reported `unverified` with `robotsBlocked: true`, and is free. Same honest user agent, same polite per-host pacing and `Crawl-delay`.
- `broken` means a 404/410/5xx answer, a name that does not resolve or a refused connection. `unverified` means the server refused a bot (401/403/429 and similar), timed out, or served a challenge page (never worked around). Social networks that block every bot are left out.
- Most-linked targets are checked first, up to `maxLinkChecksPerSite` (default 200, max 2,000). Broken outside links also appear on the page rows (`brokenExternalLinks`) and as the `broken_external_links` issue, and broken uncrawled internal targets join `broken_internal_links`.
- Every `unverified` row says why in `unverifiedReason`: `robots` (disallowed, not requested), `timeout`, `403`, `429`, `challenge` (bot-challenge page, never worked around) or `other`. The summary row counts them in `linksUnverifiedByReason`, and the run status reads like "3 unverified: 2 403, 1 timeout".
- Output: one `linkCheck` row per target (`url`, `linkKind`, `status`, `linkResult`, `unverifiedReason`, `method`, `foundOn`). Open the **Link checks** view.

### Fix list

With `includeFixList` on (default), the dataset also gets one free row per issue: `url`, `issue`, `severity`, `fixHint` (what to change), worst first. Open the **Fix list** view, export the dataset as CSV with that view, or download `FIX_LIST.csv` from the run's key-value store.

### Input

```json
{
  "websites": ["https://example.com"],
  "maxPagesPerSite": 100,
  "checkLinks": false,
  "maxLinkChecksPerSite": 200,
  "includeFixList": true,
  "requestDelaySeconds": 0.5
}
```

Leave the input empty and it audits `keencrest-guides.web.app` (our own guides site, up to 5 pages) as a demo.

### Output (one page row)

```json
{
  "rowType": "page",
  "url": "https://example.com/pricing",
  "finalUrl": "https://example.com/pricing/",
  "status": 200,
  "score": 78,
  "worstSeverity": "medium",
  "issues": [
    {"code": "redirect_chain", "severity": "medium", "message": "Redirect chain of 2 or more hops"},
    {"code": "images_missing_alt", "severity": "medium", "message": "Images without alt text"}
  ],
  "title": "Pricing | Example",
  "titleLength": 17,
  "metaDescription": "Plans and prices for Example.",
  "h1Count": 1,
  "canonical": "https://example.com/pricing/",
  "noindex": false,
  "imagesTotal": 8,
  "imagesMissingAlt": 3,
  "wordCount": 612,
  "internalLinks": 41,
  "externalLinks": 3,
  "brokenInternalLinks": [],
  "redirectCount": 2,
  "responseTimeMs": 240,
  "pageWeightBytes": 58214,
  "structuredData": ["Organization", "Product"],
  "error": null,
  "failureReason": null,
  "scrapedAt": "2026-11-01T10:00:00+00:00"
}
```

`failureReason` is `null` when the page was read as HTML, otherwise one of `dns_error`, `timeout`, `connection_error`, `blocked_403`, `rate_limited_429`, `http_<code>`, `not_html`, `robots_disallowed`, `invalid_url`, `crawl_error`, so you can count failed pages by cause without parsing `error`.

The site summary row (`rowType: "siteSummary"`, free) holds `pagesAudited`, `averageScore`, counts of pages per severity, and `issues`: the ranked table with `code`, `severity`, `pagesAffected` and up to 3 example URLs for each issue. Export the dataset as CSV or Excel for a fix list.

### Pricing

**$8.00 per 1,000 pages audited** ($0.008 a page), pay per event. The summary row and the fix list rows are free. Pages that robots.txt disallows or that cannot be reached are not billed. Worked example: a 300-page site costs $2.40; a 5,000-page site $40.

**Link check add-on: $1.00 per 1,000 links checked** ($0.001 a link), only when `checkLinks` is on. Targets that robots.txt disallows are free. Worked example: a 100-page site with 200 outside links checked costs $0.80 for pages plus $0.20 for links, $1.00 in total. `maxPagesPerSite`, `maxLinkChecksPerSite` and your run's maximum charge all cap spend; the crawl never goes past what you can be billed for.

### FAQ

**Can I audit a competitor?** Use it on sites you own, manage or have permission to audit. The crawl is polite and honours robots.txt, but the data is yours to use responsibly.

**Does it run JavaScript?** No. It audits the HTML the server returns, which is what search crawlers read first. Pages that only render content client-side will show thin content.

**Why is a broken link not listed?** Without `checkLinks`, a link is reported broken only when its target was itself audited in the same run and answered an error. Turn `checkLinks` on to test outside links and uncrawled targets too.

**Why is a link "unverified"?** The target refused a bot (many sites answer 403 or 429 to anything automated), timed out, or its robots.txt disallows the URL; the `unverifiedReason` column tells which. The Actor does not retry around it, so check those few by hand.

**Is the data personal data?** It is page metadata and link counts. No page text is stored beyond titles, descriptions and the first H1.

# Actor input Schema

## `websites` (type: `array`):

Site URLs or domains to audit, for example https://example.com. A URL with a path is used as the starting page. Only the same host is crawled.

## `maxPagesPerSite` (type: `integer`):

Upper bound on pages audited per website. Each audited page is billed, so this is also your cost cap per site.

## `checkLinks` (type: `boolean`):

Off by default. When on, the links found on your pages that point to other sites, or to your own URLs the crawl did not fetch (PDFs, images, pages past the limit), are requested with HEAD (then GET if HEAD fails) and reported as ok, broken or unverified. Each URL's robots.txt is honoured and links the target's robots.txt disallows are skipped for free. Billed per link checked.

## `maxLinkChecksPerSite` (type: `integer`):

Only used when 'Check outside and uncrawled links' is on. Upper bound on distinct link targets checked per website (most-linked first); each one checked is billed, so this is also your link-check cost cap.

## `includeFixList` (type: `boolean`):

Adds one free row per issue (url, issue, severity, fix hint) to the dataset and a FIX_LIST.csv file to the key-value store. Open the 'Fix list' view or download the CSV.

## `requestDelaySeconds` (type: `number`):

Pause between requests to the same site (randomised up to 1.3x). The Actor also honours a Crawl-delay in robots.txt and backs off on HTTP 429/503. Raise it for fragile servers.

## Actor input object example

```json
{
  "websites": [
    "https://books.toscrape.com"
  ],
  "maxPagesPerSite": 10,
  "checkLinks": false,
  "maxLinkChecksPerSite": 200,
  "includeFixList": true,
  "requestDelaySeconds": 0.5
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `fixListCsv` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "websites": [
        "https://books.toscrape.com"
    ],
    "maxPagesPerSite": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("datagleaner/seo-site-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "websites": ["https://books.toscrape.com"],
    "maxPagesPerSite": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("datagleaner/seo-site-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "websites": [
    "https://books.toscrape.com"
  ],
  "maxPagesPerSite": 10
}' |
apify call datagleaner/seo-site-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datagleaner/seo-site-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/kItusYpLNEvKMfXlf/builds/btPY4GNHB4Qo3Z90p/openapi.json