# SEO Audit and Broken Link Checker (`jakob_jerich/seo-broken-link-auditor`) Actor

Crawls a website and returns one row per page with on-page SEO data (title, meta description, H1, canonical, noindex, alt text, structured data), issue codes, and every broken internal or external link. $2 per 1,000 pages.

- **URL**: https://apify.com/jakob_jerich/seo-broken-link-auditor.md
- **Developed by:** [Jakob Jerič](https://apify.com/jakob_jerich) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit and Broken Link Checker

Crawls a website from one or more start URLs and returns one row per page with its on-page SEO data, a list of SEO issues, and every broken link found on that page. Internal and external links are checked in the same run. Plain HTTP requests (no browser), so it is fast and cheap: 150 pages of books.toscrape.com took 13 seconds.

### What it checks

**Per page:** HTTP status, redirect chain, response time, title and length, meta description and length, H1 texts, H2 count, canonical URL, meta robots and `X-Robots-Tag`, indexability, `lang`, viewport tag, word count, images and images without `alt`, internal/external link counts, hreflang tags, Open Graph title/description/image, JSON-LD and microdata types.

**Links:** every unique link on the audited pages is checked once. Internal pages are checked by crawling them; internal files (PDF, images, archives...) and external links get a HEAD request, with a GET fallback when the server rejects HEAD. A link counts as broken on HTTP 404, 410, any other 4xx except 401/403/429, any 5xx, or a connection error/timeout. Internal links that redirect are listed separately in `redirectedLinks`.

**Issue codes** (in `issues`):

| Code | Meaning |
|---|---|
| `fetch_failed`, `http_error` | Page could not be fetched, or returned 4xx/5xx |
| `missing_title`, `title_too_long`, `title_too_short`, `duplicate_title` | Title missing, over 60 or under 10 characters, or shared with another audited page |
| `missing_meta_description`, `meta_description_too_long`, `meta_description_too_short`, `duplicate_meta_description` | Same for the meta description (limits 160 / 50) |
| `missing_h1`, `multiple_h1` | No H1, or more than one |
| `missing_canonical`, `canonical_points_elsewhere` | No canonical tag, or it points to another URL |
| `noindex` | Meta robots or `X-Robots-Tag` contains `noindex` |
| `missing_lang`, `missing_viewport` | No `<html lang>`, no viewport meta tag |
| `images_missing_alt` | At least one `<img>` without an `alt` attribute (`alt=""` is fine) |
| `thin_content` | Fewer than 200 words of visible text |
| `redirected`, `slow_response` | The page was reached through a redirect, or took over 2 s |
| `broken_links`, `links_to_redirects` | The page links to broken URLs, or to internal URLs that redirect |

A site-wide summary is saved to the key-value store as `SUMMARY`: issue counts and every broken link with the pages it appears on (up to 20 per link).

### Input

```json
{
  "startUrls": [{ "url": "https://crawlee.dev" }],
  "maxPages": 20,
  "checkExternalLinks": true,
  "excludeUrlPatterns": ["/api/"],
  "maxConcurrency": 5
}
```

- `startUrls`: where to start. The crawler stays on the same host (`www.` and non-`www` count as one site). Several sites can be audited in one run.
- `maxPages`: stop after this many pages (default 100, max 10,000).
- `checkExternalLinks`: turn off to check only internal links.
- `excludeUrlPatterns`: regular expressions for URLs not to crawl (tag pages, filters, calendars).
- `maxConcurrency`: parallel page requests (default 5).

### Example output

One dataset item, copied from a real run on crawlee.dev (the three broken links point to hostnames that do not exist, such as `www.en.wikipedia.org`):

```json
{
  "url": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "finalUrl": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "statusCode": 200,
  "redirectChain": [],
  "depth": 2,
  "responseTimeMs": 73,
  "contentType": "text/html; charset=utf-8",
  "title": "Crawlee for Python v1 | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "titleLength": 79,
  "metaDescription": "Announcing the Crawlee for Python v1.0 release.",
  "metaDescriptionLength": 47,
  "h1": [
    "Crawlee for Python v1"
  ],
  "h2Count": 10,
  "canonicalUrl": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "metaRobots": null,
  "isIndexable": true,
  "lang": "en",
  "hasViewport": true,
  "wordCount": 3644,
  "imagesCount": 12,
  "imagesMissingAlt": 0,
  "internalLinksCount": 72,
  "externalLinksCount": 28,
  "hreflang": [
    {
      "lang": "en",
      "url": "https://crawlee.dev/blog/crawlee-for-python-v1"
    }
  ],
  "openGraph": {
    "title": "Crawlee for Python v1 | Crawlee for JavaScript · Build reliable crawlers. Fast.",
    "description": "Announcing the Crawlee for Python v1.0 release.",
    "image": "https://crawlee.dev/img/crawlee-js-og.png"
  },
  "structuredDataTypes": [
    "BlogPosting"
  ],
  "brokenLinks": [
    {
      "url": "https://www.en.wikipedia.org/wiki/Robots.txt",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    },
    {
      "url": "https://www.developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    },
    {
      "url": "https://www.en.wikipedia.org/wiki/Device_fingerprint",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    }
  ],
  "redirectedLinks": [],
  "issues": [
    "title_too_long",
    "meta_description_too_short",
    "broken_links"
  ],
  "issuesCount": 3,
  "error": null,
  "crawledAt": "2026-10-31T23:50:29+00:00"
}
```

### Pricing

Pay per event: **$0.002 per audited page** ($2 per 1,000 pages). Link checks are included; there is no separate charge per link or per run start. Platform usage is included in the price. You can cap the cost of a run with the "Maximum cost per run" setting; the Actor then audits only as many pages as fit.

For comparison, on 2026-10-31 other SEO audit Actors on the Store charged $0.01–$0.04 per page, and a per-link broken-link checker charged $0.001 per link (a page with 50 links costs $0.05 there).

### Use cases

- Find broken links and 404s before or after a site migration
- List pages with missing or duplicate titles and meta descriptions
- Check that important pages are indexable (no stray `noindex`, correct canonical)
- Schedule a weekly run and get alerted when `brokenLinksCount` in the summary goes up

### FAQ

**Does it respect robots.txt?** Yes. URLs disallowed for `SEOAuditBot` or `*` are not crawled (their count is in the summary), and a `Crawl-delay` is honoured (up to 10 s), in which case pages are fetched one at a time. If robots.txt itself answers with a server error, the site is not crawled.

**Why is a link marked broken that works in my browser?** Connection errors and 5xx can also come from sites that block data-center traffic or a particular country. The `statusCode` and `error` fields tell you what happened; re-check such links by hand.

**Why are links to LinkedIn or some shops not reported?** They answer 403, 429 or 999 to automated requests. Those statuses are not counted as broken, because the link usually works for people.

**Are internal pages beyond `maxPages` checked?** No. Internal pages are checked by crawling them, so raise `maxPages` to cover the whole site. Internal files (PDF, images, downloads) are always checked.

**Does it render JavaScript?** No. It reads the HTML the server sends. Sites that build their content and links only in the browser show few pages and thin content.

### Limits

- No JavaScript rendering, no login, no proxies, no anti-bot workarounds. Sites that block automated requests fail with a clear message.
- Up to 5,000 links that were not crawled as pages are checked per run.
- Results are written at the end of the run. A run that hits its timeout stops crawling early and still saves what it has.
- Core Web Vitals are not measured (that needs a real browser); `responseTimeMs` is the server response time.

# Actor input Schema

## `startUrls` (type: `array`):

Website pages to start crawling from, usually the home page. The crawler follows links on the same host (www and non-www count as the same site) and audits each page it finds.

## `maxPages` (type: `integer`):

Stop after auditing this many pages. Each audited page is one billable result. Links found on these pages are still checked.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites (one HEAD request each, GET as a fallback). Turn off to check internal links only and finish faster.

## `excludeUrlPatterns` (type: `array`):

Regular expressions. Internal URLs that match any of them are not crawled, for example "/tag/" or "?page=". Useful for skipping calendars, filters and other endless URL spaces.

## `maxConcurrency` (type: `integer`):

How many pages are fetched in parallel. Keep it low for small servers. If the site's robots.txt sets a Crawl-delay, the crawler fetches that site one page at a time instead.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPages": 20,
  "checkExternalLinks": true,
  "excludeUrlPatterns": [
    "/api/"
  ],
  "maxConcurrency": 5
}
```

# Actor output Schema

## `pages` (type: `string`):

One item per audited page with SEO fields, issues, broken links and redirected links.

## `summary` (type: `string`):

Site-wide totals: issue counts and every broken link with the pages it was found on.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://crawlee.dev"
        }
    ],
    "maxPages": 20,
    "checkExternalLinks": true,
    "excludeUrlPatterns": [
        "/api/"
    ],
    "maxConcurrency": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("jakob_jerich/seo-broken-link-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://crawlee.dev" }],
    "maxPages": 20,
    "checkExternalLinks": True,
    "excludeUrlPatterns": ["/api/"],
    "maxConcurrency": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("jakob_jerich/seo-broken-link-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPages": 20,
  "checkExternalLinks": true,
  "excludeUrlPatterns": [
    "/api/"
  ],
  "maxConcurrency": 5
}' |
apify call jakob_jerich/seo-broken-link-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,jakob_jerich/seo-broken-link-auditor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3fxqDuzTAXCBl1Wo0/builds/0O5kwVSGDUh4MYiGI/openapi.json