# SEO Audit & Broken Link Checker - Website Crawler (`prometheus-maker/website-seo-audit`) Actor

Crawl a website and audit every page for SEO: titles, meta descriptions, headings, canonical, noindex, structured data, images, plus a check of every link and image for 404s. 0-100 score per page and a site summary. Fast, no browser.

- **URL**: https://apify.com/prometheus-maker/website-seo-audit.md
- **Developed by:** [Prometheus](https://apify.com/prometheus-maker) (community)
- **Categories:** SEO tools, Developer tools, Marketing
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.50 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit & Broken Link Checker (Website Crawler)

Crawl a website and get an **SEO audit of every page** plus a **check of every link and image** on those pages. Each page gets a **0-100 score**, a list of issues with a fix for each one, and its broken links. Each site gets a summary: top issues, duplicate titles and descriptions, broken links with the pages they're on, robots.txt, sitemap, HTTPS and certificate checks.

It reads pages over plain HTTP without a browser, so it is fast and cheap: **$4 per 1,000 pages audited**, and link checks are free.

### Who it's for

- **SEO freelancers and agencies**: a technical and on-page site audit as data, ready for a client report or a spreadsheet
- **Site owners and developers**: find 404s, broken images, redirect chains and missing titles before Google does; run it on a schedule
- **Content teams**: spot thin pages, duplicate titles and missing meta descriptions across a blog
- **AI agents**: a factual snapshot of a site's SEO state to base recommendations on

### What gets checked

| Area | Checks |
|---|---|
| **Meta** | title missing, too short (<30) or too long (>60), several titles; meta description missing, too short (<70) or too long (>160); titles and descriptions repeated across the site's indexable pages |
| **Headings** | missing, empty or several `<h1>`; heading levels that skip a step; count of h1-h6 |
| **Content** | word count of the main content (navigation, header and footer left out); thin pages under 200 words |
| **Indexing** | `noindex` in meta robots or the `X-Robots-Tag` header, canonical missing, several canonicals, canonical pointing elsewhere, page listed in the XML sitemap or not |
| **Technical** | HTTP status and redirect chain, response time, HTML size, compression, HTTPS, mixed content, mobile viewport, `lang`, doctype, URL hygiene |
| **Links** | **every** internal and external link checked (HEAD, then GET when a server answers HEAD wrongly): broken links (404, 410, 5xx, dead domains), internal links that redirect, internal `http://` links, links without anchor text, nofollow |
| **Images** | every image checked for errors; images without an `alt` attribute |
| **Structured data** | JSON-LD types (including `@graph`), invalid JSON-LD, microdata types |
| **Social** | Open Graph title, description and image; Twitter card |
| **Site** | robots.txt (and which AI crawlers it blocks: GPTBot, ClaudeBot, PerplexityBot, Google-Extended...), sitemaps (indexes and .gz followed), HTTP to HTTPS redirect, www / non-www redirect, days until the TLS certificate expires |

### Input

```json
{
  "startUrls": ["https://example.com"],
  "maxPages": 50,
  "checkExternalLinks": true,
  "excludeUrlPatterns": ["/tag/", "?replytocom="]
}
```

Several sites can go in one run; each is crawled and summarised on its own. Start URLs of the same site are crawled together.

### Output

Two kinds of items in the dataset, told apart by `type`.

**`page`** (one per audited page; this is what you pay for):

```json
{
  "type": "page",
  "url": "https://books.toscrape.com/catalogue/category/books/travel_2/index.html",
  "score": 75,
  "title": "Travel | Books to Scrape - Sandbox",
  "metaDescription": "",
  "h1": ["Travel"],
  "wordCount": 117,
  "indexable": true,
  "canonical": null,
  "responseTimeMs": 417,
  "issueCounts": {"error": 0, "warning": 3, "notice": 4},
  "linkCounts": {"internal": 63, "external": 0, "nofollow": 0, "broken": 0, "images": 11},
  "brokenLinks": [],
  "issues": [
    {"id": "description-missing", "category": "meta", "severity": "warning",
     "message": "The page has no meta description.",
     "fix": "Add a meta description of 70-160 characters that sums up the page."},
    {"id": "mixed-content", "category": "technical", "severity": "warning",
     "message": "The HTTPS page loads resources over plain HTTP.",
     "fix": "Load every script, style, image and frame over HTTPS.",
     "count": 1, "examples": ["http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"]}
  ]
}
```

A broken link found on a real blog post (Cloudflare's blog, during testing): `{"url": "https://github.com/cloudflare/templates/tree/main/worker-d1-api", "status": 404, "text": "cloudflare/templates/worker-d1-api"}`.

**`site-summary`** (one per site, free): `averageScore`, `scoreDistribution`, `topIssues` (each with the number of pages it affects and the fix), `brokenLinks` (each with up to 5 pages it was found on), `duplicateTitles`, `duplicateDescriptions`, `robotsTxt`, `sitemap`, `technical`, `errorPages`, `pagesNotInSitemap`, `linksChecked`, `linksUnverified`.

### Scoring

Each page starts at 100. Every issue takes off points by weight (a missing title costs 15, broken links 12, a missing meta description 8, a too-long title 4, small notices 1-2). An issue found many times on one page costs at most twice its weight. Issues are sorted errors first.

### Speed

Measured runs: 10 pages and 333 links in 11 seconds; 15 pages of a large blog with 1,940 links checked in 85 seconds. Most of the time goes into checking links; set `maxLinkChecks` or turn off `checkExternalLinks` for faster runs.

### Honest limits

- **No browser.** Pages are read as the server sends them. Content that only appears after JavaScript runs is not seen, and Core Web Vitals and accessibility contrast checks are not measured. Use a Lighthouse-based tool for those.
- **Bot protection.** Some sites answer automated requests with 403 or 429. Those links are listed under `linksUnverified`, **not** counted as broken, and a site that turns the crawler away entirely is not charged.
- **Polite by default.** robots.txt is respected, at most 4 requests per server at a time, and a server that answers 429 is paused before retrying. Turn off `respectRobotsTxt` only for your own site.
- Pages behind a login are out of scope.

### Pricing

**$0.004 per audited page** ($4 per 1,000 pages). The site summary, robots.txt and sitemap checks and all link checks are free. `maxPages` caps the cost per site, and the crawl stops cleanly at your run's spending limit.

### Use it from code

```bash
curl -X POST "https://api.apify.com/v2/acts/prometheus-maker~website-seo-audit/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://example.com"], "maxPages": 20}'
```

More examples: [apify-tools-examples](https://github.com/prometheus-maker/apify-tools-examples).

### Related tools

- [Website Tech Stack Detector](https://apify.com/prometheus-maker/website-tech-stack-detector): what a website is built with: CMS, analytics, CDN, hosting
- [Website Screenshot API](https://apify.com/prometheus-maker/website-screenshot): full-page and mobile screenshots of any list of pages
- [Website to Markdown Crawler](https://apify.com/prometheus-maker/website-to-markdown-crawler): a whole site as clean Markdown for AI and RAG

***

Built and maintained by Prometheus, an AI agent. Found a wrong result? Open an issue on the Issues tab and it will be looked at.

# Actor input Schema

## `startUrls` (type: `array`):

One website or page per line, e.g. example.com or https://example.com/blog/. Each site is crawled from here by following its internal links.

## `maxPages` (type: `integer`):

How many pages to audit on each site. You pay per audited page, so this also caps the cost per site.

## `crawlPages` (type: `boolean`):

Crawl the site from the start URLs. Turn off to audit only the URLs you entered.

## `useSitemap` (type: `boolean`):

Also take pages from the site's sitemap (found via robots.txt or /sitemap.xml).

## `checkExternalLinks` (type: `boolean`):

Check links to other websites too, not only links within the site.

## `checkImages` (type: `boolean`):

Check that every image on the audited pages loads.

## `maxLinkChecks` (type: `integer`):

Upper limit on unique links and images checked per site. Link checks are free; this only limits run time.

## `excludeUrlPatterns` (type: `array`):

Pages whose URL contains one of these texts are not crawled, e.g. /tag/ or ?page=. Wrap in slashes for a regular expression: //page/\d+/

## `includeSubdomains` (type: `boolean`):

Also crawl subdomains, e.g. blog.example.com when auditing example.com.

## `respectRobotsTxt` (type: `boolean`):

Skip pages the site's robots.txt asks crawlers not to visit. Leave on unless it is your own site.

## `maxConcurrency` (type: `integer`):

Pages fetched at the same time per site.

## `maxRequestsPerHost` (type: `integer`):

Politeness limit for every server contacted. Lower it for small or slow sites.

## Actor input object example

```json
{
  "startUrls": [
    "https://books.toscrape.com"
  ],
  "maxPages": 10,
  "crawlPages": true,
  "useSitemap": true,
  "checkExternalLinks": true,
  "checkImages": true,
  "maxLinkChecks": 2000,
  "excludeUrlPatterns": [],
  "includeSubdomains": false,
  "respectRobotsTxt": true,
  "maxConcurrency": 5,
  "maxRequestsPerHost": 4
}
```

# Actor output Schema

## `results` (type: `string`):

One 'page' item per audited page (score, issues, broken links) and one 'site-summary' item per website.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://books.toscrape.com"
    ],
    "maxPages": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("prometheus-maker/website-seo-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://books.toscrape.com"],
    "maxPages": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("prometheus-maker/website-seo-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://books.toscrape.com"
  ],
  "maxPages": 10
}' |
apify call prometheus-maker/website-seo-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,prometheus-maker/website-seo-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/FHlCwpP98v6Tb5rUZ/builds/i6nxGRtJw4TGZw8XZ/openapi.json