# SEO Audit Crawler: Site Audit, Broken Links & On-Page SEO (`jobdata_labs/seo-audit-crawler`) Actor

Crawl a website and audit every page: titles, meta descriptions, H1, canonicals, noindex, broken links, redirects, alt text, structured data, speed. Per-page score plus an HTML report.

- **URL**: https://apify.com/jobdata_labs/seo-audit-crawler.md
- **Developed by:** [Momi](https://apify.com/jobdata_labs) (community)
- **Features:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.10 / 1,000 audited pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit Crawler: Site Audit, Broken Links & On-Page SEO

Crawl any website and get a **page-by-page SEO audit** with a score from 0 to 100, a list of concrete issues for every page, and a ready-to-share **HTML report**. Built for SEO consultants, agencies, web developers and site owners who need a fast technical check without installing desktop software.

### What does SEO Audit Crawler do?

It starts from a URL, follows internal links like a search engine would, and checks every HTML page for:

- **Status codes**: 4xx and 5xx pages, redirect chains.
- **Titles**: missing, too short, too long, duplicated across pages.
- **Meta descriptions**: missing, too short, too long, duplicated.
- **Headings**: missing or multiple H1.
- **Indexing**: noindex in meta robots or X-Robots-Tag, canonical pointing elsewhere, missing canonical.
- **Links**: broken internal links (with the pages that contain them), optional broken external links.
- **Images**: images without alt text.
- **Mobile & security**: missing viewport, plain HTTP, mixed content.
- **Content & sharing**: thin content (under 200 words), missing Open Graph, missing JSON-LD structured data (detected types are listed).
- **Speed**: server responses slower than your threshold.
- **International SEO**: hreflang alternates and the html lang attribute.

### Why use it?

- **Pay only for audited pages**: no subscription. A 100-page audit costs a few cents (see the pricing tab).
- **Actionable output**: pages are sorted from worst to best score, each issue has a severity (error, warning, notice) and details such as the broken URLs or the measured length.
- **Client-ready report**: an HTML summary with the average score, issue totals and the pages to fix first.
- **Automate it**: schedule weekly audits and send results to Google Sheets, Slack or email through Apify integrations, or call it from your own code and AI agents.
- **Polite crawling**: own user agent, robots.txt respected, adjustable concurrency.

### How to use it

1. Enter the **Website URL** (for example `https://example.com`).
2. Set **Max pages** (start with 50 to 200 for a first look).
3. Optionally turn on **Check external links**.
4. Click **Start**. When the run finishes, open the **HTML audit report** in the Output tab, or export the dataset to Excel, CSV or JSON.

### Input

| Field | Default | Description |
| :--- | :--- | :--- |
| `startUrl` | required | Page to start from. |
| `maxPages` | 100 | Pages to audit; also caps the cost. |
| `maxDepth` | 5 | Clicks away from the start URL. |
| `includeSubdomains` | false | Also crawl subdomains. |
| `excludePatterns` | \[] | Skip URLs containing these fragments. |
| `checkExternalLinks` | false | Check outgoing links too. |
| `maxExternalLinks` | 200 | Limit of external URLs to check. |
| `titleMin` / `titleMax` | 30 / 60 | Title length thresholds. |
| `descriptionMin` / `descriptionMax` | 70 / 160 | Meta description thresholds. |
| `slowPageMs` | 2000 | Slow response threshold in milliseconds. |
| `maxConcurrency` | 5 | Parallel requests (1 to 10). |
| `respectRobotsTxt` | true | Skip URLs disallowed by robots.txt. |

### Output

One dataset row per page (worst pages first):

```json
{
  "url": "https://example.com/blog",
  "status": 200,
  "score": 79,
  "title": "Blog",
  "titleLength": 4,
  "metaDescription": null,
  "h1": ["Latest posts"],
  "canonical": "https://example.com/blog",
  "wordCount": 412,
  "imagesMissingAlt": 3,
  "structuredDataTypes": ["BreadcrumbList"],
  "responseTimeMs": 380,
  "inboundInternalLinks": 12,
  "issues": [
    { "code": "TITLE_TOO_SHORT", "severity": "warning", "message": "Title is shorter than the recommended minimum.", "details": "4 characters" },
    { "code": "MISSING_DESCRIPTION", "severity": "warning", "message": "Missing meta description." },
    { "code": "IMAGES_MISSING_ALT", "severity": "warning", "message": "Images without alt text.", "details": "3 of 8 images" }
  ]
}
```

The **Issues** dataset view lists one row per issue, handy for spreadsheets. The key-value store also contains `SUMMARY` (JSON) and `REPORT` (HTML).

### Use it from code or AI agents

```js
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('jobdata_labs/seo-audit-crawler').call({ startUrl: 'https://example.com', maxPages: 100 });
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

The Actor is also available to AI agents through the Apify MCP server.

### Scoring

Each page starts at 100. Every error removes 15 points, every warning 6, every notice 2 (minimum 0). Thresholds for titles, descriptions and speed are configurable.

### FAQ

**Does it render JavaScript?**
No. It reads the HTML returned by the server, like most SEO crawlers in their default mode. Sites that build all content in the browser will show thin content.

**Can I audit a site I don't own?**
Yes, for public pages. The crawler identifies itself and respects robots.txt. Keep concurrency low on small sites.

**Why is an external link marked broken?**
Only clear failures count: 404, 410, 5xx or an unreachable host. Sites that block bots with 401, 403 or 429 are not reported.

**How is this different from a sitemap tool?**
It follows real links and inspects each page's content. To track which pages appear or disappear over time, use [Sitemap Change Monitor](https://apify.com/jobdata_labs/sitemap-change-monitor).

# Actor input Schema

## `startUrl` (type: `string`):

Home page or any page of the site to audit, for example https://example.com. The crawler follows internal links from here.

## `maxPages` (type: `integer`):

Maximum number of pages to crawl and audit. You pay per audited page, so this also caps the cost. Set to 0 for no limit.

## `maxDepth` (type: `integer`):

How many clicks away from the start URL the crawler may go.

## `includeSubdomains` (type: `boolean`):

Also crawl subdomains such as blog.example.com. By default only the start host (with or without www) is crawled.

## `excludePatterns` (type: `array`):

Skip internal URLs that contain any of these text fragments, for example /tag/ or ?sort=.

## `checkExternalLinks` (type: `boolean`):

Also request outgoing links to other websites and report the broken ones (404, 410, 5xx, unreachable). Sites that block bots (401, 403, 429) are not reported as broken.

## `maxExternalLinks` (type: `integer`):

Upper limit of distinct external URLs checked per run.

## `titleMin` (type: `integer`):

Titles shorter than this many characters are flagged.

## `titleMax` (type: `integer`):

Titles longer than this many characters are flagged as likely to be truncated in search results.

## `descriptionMin` (type: `integer`):

Meta descriptions shorter than this many characters are flagged.

## `descriptionMax` (type: `integer`):

Meta descriptions longer than this many characters are flagged.

## `slowPageMs` (type: `integer`):

Pages whose server response takes longer than this are flagged.

## `maxConcurrency` (type: `integer`):

Number of pages requested in parallel (1 to 10). Keep it low for small or shared servers.

## `respectRobotsTxt` (type: `boolean`):

Skip URLs that the site's robots.txt disallows. Turn off only for your own sites.

## Actor input object example

```json
{
  "startUrl": "https://crawlee.dev",
  "maxPages": 20,
  "maxDepth": 5,
  "includeSubdomains": false,
  "checkExternalLinks": false,
  "maxExternalLinks": 200,
  "titleMin": 30,
  "titleMax": 60,
  "descriptionMin": 70,
  "descriptionMax": 160,
  "slowPageMs": 2000,
  "maxConcurrency": 5,
  "respectRobotsTxt": true
}
```

# Actor output Schema

## `report` (type: `string`):

No description

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://crawlee.dev",
    "maxPages": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("jobdata_labs/seo-audit-crawler").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrl": "https://crawlee.dev",
    "maxPages": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("jobdata_labs/seo-audit-crawler").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://crawlee.dev",
  "maxPages": 20
}' |
apify call jobdata_labs/seo-audit-crawler --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,jobdata_labs/seo-audit-crawler"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/W9fOanTLRSi6hmwSd/builds/NjYaKdzXR4CDimnGZ/openapi.json