# SEO Site Audit & Broken Link Checker: Technical SEO Crawler (`earlytern/seo-site-audit`) Actor

Crawl any website and audit every page for technical SEO issues: titles, meta descriptions, H1, canonicals, noindex, broken links, redirects, duplicates, orphan pages, sitemap gaps and AI crawler blocking. Obeys robots.txt.

- **URL**: https://apify.com/earlytern/seo-site-audit.md
- **Developed by:** [earlytern](https://apify.com/earlytern) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Site Audit & Broken Link Checker: Technical SEO Crawler

Crawl a whole website and get a **technical SEO audit of every page** in one table: title and meta description
(missing, too short, too long, duplicated), H1 headings, canonical tags, noindex, word count, images without alt text,
structured data, response time, redirects and **broken links**. Each website also gets a **site summary**: pages per
issue, every broken link with how many pages carry it, redirect chains, duplicate titles, **orphan pages**, pages
missing from the sitemap, and which **AI crawlers** the site blocks. Audit 1 website or 100 in one run.

### What does SEO Site Audit do?

- **Crawls the site like a search engine**: starts at the address you give, follows internal links, then reads the
  XML sitemap (also gzip and sitemap indexes) and audits the pages only the sitemap lists.
- **Audits every page** for 21 common technical SEO issues (full list below) and gives the raw values too, so you can
  apply your own rules.
- **Checks every link once**: internal links, links to files (PDF, images) and links to other websites, and reports
  each broken one (404, 410, 5xx, dead domains) on the pages that carry it.
- **Compares the crawl with the sitemap**: orphan pages (in the sitemap, linked from nowhere), sitemap addresses the
  crawl never reached, and indexable pages missing from the sitemap.
- **AI search readiness**: lists the well-known AI crawlers the site's robots.txt blocks, and whether it has an
  `llms.txt` file.
- Follows redirects step by step and reports **redirect chains** (two or more hops).
- Fast and light: plain HTTP requests, no browser, no proxy.
- **Polite by design**: always obeys robots.txt (including Crawl-delay), waits between requests to each website,
  slows down when a site asks it to, and never logs in.

### Who uses it?

- **SEO agencies and freelancers**: run a first audit for a new client, then schedule it monthly and compare.
- **Website owners and marketers**: find broken links, missing titles and descriptions before they cost traffic.
- **Developers and QA teams**: check a site after a migration or a release for 404s, redirect chains and noindex tags
  left by mistake.
- **Content teams**: spot thin pages, duplicate titles and pages no other page links to.

### How to use it

1. Click **Try for free**.
2. In **Start URLs**, add the home page of each website to audit, for example `https://www.example.com`.
3. Set **Maximum pages per site** (the default is 100). You pay only for pages audited.
4. Click **Start**. Download the results as JSON, CSV or Excel, or use the API. The **Pages** view lists every page
   with its issues; the **Site summaries** view gives one row per website.

Options: turn off **Check links** for a faster run, turn off **Check links to other websites** to check only your
own links, tick **Include subdomains** to audit `blog.example.com` together with `example.com`, and use **Exclude
URLs containing** to skip sections such as `/tag/` or `?sort=`.

### Use with AI agents (MCP)

AI assistants and AI agents can call this Actor through Apify's MCP server at [mcp.apify.com](https://mcp.apify.com)
or through the Apify API. Questions it answers well:

- "Does example.com have broken links?" Read `brokenLinks` in the site summary row: each broken link, its status and
  how many pages carry it.
- "Which pages of example.com have no meta description?" Read the page rows whose `issues` contain
  `missing-meta-description`.
- "Is example.com ready for AI search?" Read `blockedForAiBots` and `hasLlmsTxt` in the site summary row.

Only `startUrls` is required. Keep **Maximum pages per site** low (for example 20) for quick answers, and read the
site summary row first: it sums up the whole audit in one item.

### Input example

```json
{
  "startUrls": [{ "url": "https://www.example.com" }],
  "maxPagesPerSite": 500,
  "checkLinks": true,
  "checkExternalLinks": true,
  "includeSubdomains": false,
  "excludeUrlPatterns": ["/tag/", "?sort="],
  "maxConcurrency": 5
}
```

### Output example

Real rows from a test run on 2 November 2026 (lists shortened). One **page** row:

```json
{
  "recordType": "page",
  "site": "https://crawlee.dev/",
  "url": "https://crawlee.dev/blog",
  "statusCode": 200,
  "depth": 1,
  "redirectChain": [],
  "title": "Crawlee Blog - learn how to build better scrapers | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "titleLength": 107,
  "metaDescription": "Guides and tutorials on using Crawlee, the most reliable open-source web scraping and browser automation library for JavaScript and Node.js developers.",
  "metaDescriptionLength": 151,
  "h1": null,
  "h1Count": 0,
  "canonical": "https://crawlee.dev/blog",
  "indexable": true,
  "wordCount": 1573,
  "imagesMissingAlt": 0,
  "internalLinks": 40,
  "externalLinks": 29,
  "inboundLinks": 4,
  "brokenLinks": [],
  "schemaTypes": ["Blog"],
  "responseTimeMs": 988,
  "issues": ["title-too-long", "missing-h1"],
  "crawledAt": "2026-11-02T15:35:46.893Z"
}
```

One **site** summary row (a 5-page test, so the crawl stopped at the page limit):

```json
{
  "recordType": "site",
  "site": "https://crawlee.dev/",
  "pagesCrawled": 5,
  "pagesAudited": 5,
  "pagesFailed": 0,
  "issueCounts": { "duplicate-meta-description": 3, "canonical-to-other-url": 1, "title-too-long": 2, "missing-h1": 1 },
  "duplicateTitles": [],
  "duplicateDescriptions": [
    {
      "text": "Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.",
      "urls": ["https://crawlee.dev/", "https://crawlee.dev/js"]
    }
  ],
  "redirectChains": [],
  "brokenLinks": [],
  "linksChecked": 121,
  "linksNotChecked": 8,
  "orphanPages": null,
  "sitemapOnly": null,
  "crawlOnly": [],
  "sitemapUrlCount": 4849,
  "blockedForAiBots": [],
  "hasLlmsTxt": true,
  "stoppedBy": "page-limit",
  "message": "Stopped at the maximum pages per site. Raise it to audit more of the site.",
  "crawledAt": "2026-11-02T15:36:07.603Z"
}
```

`orphanPages` and `sitemapOnly` are filled only when the crawl reached and read every page linked from the start page
(they stay empty when it stopped at the page limit, as above, or when a linked page did not answer). `crawlOnly` stays
empty when some sitemap files could not be read. The **OUTPUT** record of each run gives a short summary per
website.

#### Issues checked

| Issue | Meaning |
|---|---|
| `missing-title`, `title-too-short`, `title-too-long` | No title, or shorter than 30 / longer than 60 characters |
| `duplicate-title` | Same title as another indexable page |
| `missing-meta-description`, `meta-description-too-short`, `meta-description-too-long` | No description, or shorter than 70 / longer than 160 characters |
| `duplicate-meta-description` | Same description as another indexable page |
| `missing-h1`, `multiple-h1` | No H1 heading, or more than one |
| `missing-canonical`, `canonical-to-other-url` | No canonical tag, or one that points to another address |
| `noindex` | Kept out of search results by a robots meta tag or header |
| `low-word-count` | Fewer than 200 words of visible text |
| `images-missing-alt` | Images without an alt attribute |
| `broken-links` | Links to pages or files that answer 4xx or 5xx, or to dead domains |
| `orphan-page` | In the sitemap, but no crawled page links to it |
| `broken-page`, `server-error` | The page itself answers 4xx or 5xx |
| `redirect-chain` | Reached through two or more redirects |
| `slow-response` | Took more than 3 seconds to load |

### How much does it cost?

You pay only for pages audited. No start fee. Prices per 1,000 pages, by Apify plan:

| Event | Free | Bronze | Silver | Gold and up |
|---|---|---|---|---|
| Page audited | $6.00 | $5.00 | $4.50 | $4.00 |

- **Free**: the site summary rows, pages that answer with an error (404, 500…), and every link check.
- Pages over your maximum, pages blocked by robots.txt and excluded pages are not opened and not charged.
- Set a **maximum cost per run** and the Actor stops cleanly before going over it; the site summary says so.

Example on the Free plan: auditing a 500-page website costs $3.00; a monthly audit of 10 client sites of 200 pages
each costs $12.00.

### FAQ

**Does it run JavaScript?** No. It reads the HTML the server sends, as most search engine crawlers do first. On sites
that build their pages with JavaScript in the browser, pages may look empty (no title, few words); that is a real SEO
risk worth knowing about.

**Why was my site not crawled?** The site summary row says why in `message`: robots.txt does not allow crawlers, the
site could not be reached, or it belongs to a large platform that does not allow automated access (social networks,
search engines, large marketplaces), which this Actor never crawls.

**Why do some links count as "not checked"?** Links blocked by the target site's robots.txt, links to large platforms,
links that answer 401, 403 or 429, links over the link limits (see Limits) and links left when the run ran short of
time are not checked. They are counted in `linksNotChecked`, never reported as broken.

**Are 403 answers broken links?** No. 401, 403 and 429 usually mean a login wall, bot protection or a rate limit, so
such links are counted as not checked, never reported as broken.

**Can I audit a staging site?** Only if it is on the public internet with a normal domain name. The Actor does not
open private network addresses, IP addresses or non-standard ports, and does not log in.

### Limits

- Up to 100 websites per run and 50,000 pages per website.
- Up to 10 pages of one website at a time (default 5), with a pause between requests. A robots.txt Crawl-delay is
  always obeyed in full, so a website that asks for a long delay gets fewer pages audited before the run timeout.
- Link checks: up to 20 links per audited page (at least 500 per site, at most 50,000), and up to 200 links to any
  one other website.
- Sitemaps: up to 50 sitemap files and 200,000 addresses per website.
- Pages larger than 5 MB are read up to 5 MB, and up to 5,000 links of one page are followed and checked.
- The crawl stops in time to save its results before the run timeout; for large sites give the run more time.

### Is it legal?

Audit your own websites, your clients' websites, or websites whose owners allow it. The Actor reads only public pages,
always obeys robots.txt, never logs in and never tries to get past blocking. It keeps only the audit fields shown
above: no page text is stored, and email addresses and phone numbers found in titles, descriptions and headings are
removed. You are responsible for how you use it and for the terms of the websites you audit.

### Questions and feedback

Open an issue on the **Issues** tab.

# Actor input Schema

## `startUrls` (type: `array`):

Websites to audit, one per line, usually the home page. Each website is crawled from this address and audited separately. Audit your own websites or websites you are allowed to check.

## `maxPagesPerSite` (type: `integer`):

The crawl stops after this many pages of each website. You pay only for pages audited.

## `checkLinks` (type: `boolean`):

Also check every unique link the crawl does not open as a page (files, pages over the maximum, links to other websites), once each, to find broken links. No extra charge; the run takes longer.

## `checkExternalLinks` (type: `boolean`):

Include links that point to other websites when checking links. Links to large social networks, search engines and marketplaces are never checked.

## `includeSubdomains` (type: `boolean`):

Crawl subdomains as part of the website, for example blog.example.com when you start at example.com.

## `excludeUrlPatterns` (type: `array`):

Pages whose address contains any of these texts are neither crawled nor checked, for example /tag/ or ?sort=. Use \* for any text, for example /blog/\*/comments.

## `maxConcurrency` (type: `integer`):

How many pages of one website are read at the same time. The crawl also waits as long as the website's robots.txt asks (Crawl-delay).

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPagesPerSite": 5,
  "checkLinks": true,
  "checkExternalLinks": true,
  "includeSubdomains": false,
  "excludeUrlPatterns": [],
  "maxConcurrency": 5
}
```

# Actor output Schema

## `pages` (type: `string`):

No description

## `sites` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://crawlee.dev"
        }
    ],
    "maxPagesPerSite": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("earlytern/seo-site-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://crawlee.dev" }],
    "maxPagesPerSite": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("earlytern/seo-site-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPagesPerSite": 5
}' |
apify call earlytern/seo-site-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,earlytern/seo-site-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/HHqBDtq3mj9opCgFd/builds/YuOaq09MyKwYInIIO/openapi.json