# SEO Audit & Broken Link Checker: Site Crawler, 404s, Redirects (`stellalytics/seo-site-audit`) Actor

Technical SEO site audit crawler (SEO spider): broken links and 404s, redirect chains, missing or duplicate titles and meta descriptions, noindex and canonical problems, and orphan sitemap pages. 37 checks per page, a health score, and alerts on what changed since the last run. Obeys robots.txt.

- **URL**: https://apify.com/stellalytics/seo-site-audit.md
- **Developed by:** [Abir Riahi](https://apify.com/stellalytics) (community)
- **Categories:** SEO tools, Developer tools
- **Features:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit & Broken Link Checker: technical SEO site crawler for 404s, redirects and change alerts

**Run a technical SEO site audit on a whole website and get every problem as data.** The crawler works like an SEO spider and finds broken links, redirect chains, 4xx and 5xx pages, missing or duplicate titles and meta descriptions, noindex and canonical problems, orphan pages in the sitemap, and 37 checks in total. Each page gets its issues ranked by severity, each site gets a health score, and scheduled runs tell you what broke or got fixed since the last audit. Made for SEO agencies, web developers and site owners who audit many sites or need audits in a pipeline instead of a desktop app.

No browser, no login, no API key: the Actor reads public pages over plain HTTP, obeys robots.txt, and keeps a polite pace.

### What you get

One row per page:

| Field | Example | Meaning |
| --- | --- | --- |
| `url`, `statusCode` | `https://example.com/pricing`, `200` | The page and its HTTP status (redirects are rows too, with `redirectTo` and `redirectChain`) |
| `issues` | `["broken-internal-links", "duplicate-title", "missing-h1"]` | Every problem found, errors first |
| `errors`, `warnings`, `notices` | `1`, `1`, `1` | How many issues of each severity |
| `indexable` | `true` | 200, HTML, not noindex and not canonicalized elsewhere |
| `brokenLinks` | `[{ "url": "https://example.com/old", "status": 404, "anchor": "Old offer" }]` | Links on this page that return 4xx/5xx or fail |
| `linksToRedirects` | `[{ "url": "/team", "redirectsTo": "https://example.com/about/team" }]` | Internal links you should update to their final URL |
| `title`, `titleLength`, `metaDescription`, `metaDescriptionLength`, `h1` | | On-page basics |
| `canonical`, `noindex`, `metaRobots`, `xRobotsTag`, `hreflang`, `lang` | | Indexing signals |
| `depth`, `foundOn`, `inlinks`, `inSitemap` | `2`, `https://example.com/blog`, `14`, `true` | Clicks from the start page, where the crawler found the page, how many crawled pages link to it, and whether the sitemap lists it |
| `wordCount`, `images`, `imagesWithoutAlt`, `structuredData`, `openGraph`, `viewport` | `840`, `12`, `3`, `["Article", "BreadcrumbList"]` | Content and markup |
| `responseTimeMs`, `sizeKb` | `310`, `48.2` | Server response time and HTML size |
| `newIssues`, `fixedIssues`, `alerts` | `["http-4xx"]`, `["missing-title"]`, `["Page now fails (404); it returned 200 last run."]` | What changed since the last run of the same history |

A **SUMMARY** record in the run's key-value store gives, per site, the health score (share of pages without errors), status codes, every issue type with the number of pages, examples and a short how-to-fix hint, all broken links with the pages that link to them, duplicate title groups, orphan pages, the sitemaps read, which search engines and AI crawlers (Googlebot, Bingbot, OAI-SearchBot, GPTBot, ClaudeBot, PerplexityBot and more) robots.txt lets read the site, and changes since the last run. A **REPORT** record in the same store is a ready-to-share HTML page (health score, issues ranked by severity with how-to-fix hints, broken links with the pages that link to them, duplicate titles, AI crawler access and changes since the last run): open it from the run's **Output** tab, then save it or send it to a client. The dataset has five views: Overview, Titles/descriptions/headings, Broken links and redirects, Indexing and sitemap, and Changes since last run.

#### Sample output

The home page row from a real run on 2026-11-12 against crawler-test.com, a site built with deliberate SEO errors for testing crawlers (lists shortened):

```json
{
  "url": "https://crawler-test.com/",
  "statusCode": 200,
  "indexable": true,
  "issues": [
    "broken-internal-links",
    "broken-external-links",
    "duplicate-title",
    "missing-viewport",
    "mixed-content",
    "links-to-redirects",
    "meta-description-too-short",
    "missing-lang"
  ],
  "errors": 1,
  "warnings": 4,
  "notices": 3,
  "title": "Crawler Test Site",
  "titleLength": 17,
  "metaDescriptionLength": 40,
  "brokenLinkCount": 59,
  "brokenLinks": [
    {
      "url": "https://crawler-test.com/status_codes/status_404",
      "status": 404,
      "error": null,
      "anchor": "404 HTTP Status"
    }
  ],
  "linksToRedirects": [
    {
      "url": "https://crawler-test.com/redirects/redirect_1",
      "redirectsTo": "https://crawler-test.com/redirects/redirect_target"
    }
  ],
  "depth": 0,
  "inlinks": 164,
  "inSitemap": true,
  "responseTimeMs": 25
}
```

### The 37 checks

- **Errors:** 4xx pages, 5xx pages, pages that could not be fetched, broken internal links, redirect loops, canonical pointing to an error page, missing title.
- **Warnings:** duplicate titles, titles over 60 or under 15 characters, multiple title tags, missing or duplicate meta descriptions, missing H1, images without alt, mixed content (HTTP resources on HTTPS pages), missing viewport, broken external links, redirect chains of 2+ hops, canonical pointing to a redirect, slow responses (over 1.5 s), orphan pages (in the sitemap but not linked), sitemap URLs that redirect or fail, noindex pages listed in the sitemap, invalid JSON-LD.
- **Notices:** long or short meta descriptions, multiple H1s, thin content (under 200 words), missing lang, internal links to redirects, noindex pages, canonicalized pages, HTML over 1 MB, pages more than 4 clicks deep, meta refresh redirects, hreflang without a self-reference.

The full list with descriptions is also in the SUMMARY record (`issueTypes`).

### Use cases

- **Technical SEO audit** of a new client's site: sort by `errors` and fix the worst pages first.
- **Broken link checker** before and after a site migration or redesign.
- **Weekly site health monitoring** for agencies: one saved task per client on a schedule, and `alerts` tell you which pages started failing or became noindex.
- **Sitemap QA:** find sitemap URLs that redirect, return errors or are noindex, and pages that no link reaches.
- **Content QA in CI:** call the Actor from your pipeline after a deploy and fail the build if `errors` go up.

### How to use it

1. Paste one or more **websites** (bare domains get `https://`).
2. Set **Maximum pages per site** (each audited page is one charged result).
3. Run it. For monitoring, add a weekly **schedule**: each run is compared with the previous one.

The crawler starts at your page, follows links on the same site (www. and non-www count as one site), then audits pages that only the XML sitemap knows about. Links to other sites, files such as PDFs, and internal pages beyond your page limit are checked with one lightweight request each (not charged).

**Speed:** requests to your site start at most about 6 per second, with up to 4 in flight, so 200 pages with their link checks usually take 1 to 4 minutes. A `Crawl-delay` in robots.txt is honored (up to 10 seconds), which makes the crawl slower.

### Pricing

$0.005 per audited page ($5 per 1,000; $0.0045 on Apify's Silver plan, $0.004 on Gold and above; see the Pricing tab), plus Apify's standard start fee of $0.00005 per run. Link checks, the SUMMARY record and the change history are included with no per-link charge. Pages that could not be fetched at all (timeout, DNS or TLS error), pages behind a bot challenge or in a redirect loop, and start URLs that are not valid websites have no per-page charge. Set a **maximum cost per run** in Apify and the Actor audits only as many pages as that budget allows.

### Automate it: schedules and integrations

- **Weekly monitoring:** in Apify Console, open **Schedules**, create a weekly schedule and add this Actor or a saved task. Use a different **History name** per client so their changes are tracked separately.
- **Send results where your team works:** under the **Integrations** tab you can connect Google Sheets, Slack, email, Make, Zapier, n8n or a webhook.
- **API:** start runs and download results with the Apify API or the official Python and JavaScript clients (see the **API** tab).
- **AI agents (MCP):** connect Claude, Cursor or any MCP client to Apify's MCP server at `https://mcp.apify.com?tools=stellalytics/seo-site-audit`.

### Related Actors

- [Bulk PageSpeed & Core Web Vitals Checker](https://apify.com/stellalytics/core-web-vitals-pagespeed-checker): Lighthouse scores and real-user Core Web Vitals for the pages this audit finds.
- [Tech Stack & Domain Lookup](https://apify.com/stellalytics/website-domain-intelligence): tech stack, DNS, email setup, SSL expiry and AI-crawler rules for a list of domains.
- [AI Search Visibility Tracker](https://apify.com/stellalytics/ai-search-visibility-tracker): whether ChatGPT, Perplexity, Claude and Grok mention and cite your brand.

### How it works, and what it does not do

- It identifies itself as `SiteAuditBot` with a link to this page, reads robots.txt on every host before requesting anything there, and obeys it. If robots.txt answers with a server error or 429, or cannot be reached at all, that host is treated as fully disallowed (RFC 9309) and nothing is crawled.
- It never logs in, never fills forms, and never works around CAPTCHAs or bot protection. A page behind a bot challenge is reported as such.
- It reads HTML as the server sends it and does not run JavaScript. Sites that build their content only in the browser show little text and few links (this is also how many crawlers see them).
- It only connects to public internet addresses and refuses private networks.
- It saves facts about pages (status, tags, counts and links), not copies of their content.

### Limitations

- Up to 10,000 pages per site and 100 sites per run. Very large sites are best audited by section (start at a section URL and use **Skip URLs containing** for the rest).
- Pages only reachable through JavaScript, forms or a login are not found.
- Some servers refuse automated requests (401, 403, 429, or LinkedIn's 999), and some links time out. Those links are reported as unverified, not broken, and a page that times out or is rate-limited never raises a change alert. A link counts as broken only on a real 4xx/5xx answer, a domain that does not exist, a refused connection or an invalid certificate.
- Orphan pages are reported only when the crawl finished the whole site within the page limit, so a cut-short crawl does not flag false orphans.
- Changes in duplicate titles, duplicate descriptions and orphan pages are compared only when both runs covered the whole site, because a crawl cut short by the page limit may see a different set of pages.
- Title and description lengths are counted in characters, a common approximation of what search engines show.

### FAQ

**Can I audit a site I don't own?** The Actor only reads public pages and obeys each site's robots.txt, like a search engine crawler. Auditing competitors' public pages is common SEO practice; respect the site's terms and don't audit at a pace the site can't handle (the defaults are gentle).

**Is personal data collected?** The Actor saves technical facts about pages, not page content. Titles, headings and link anchor texts are stored as they appear on the page, so a page that names a person in its title will show that title.

**Why does a page show `orphan-page`?** The sitemap lists it, but none of the crawled pages links to it, so visitors and search engines can only find it through the sitemap.

**Why are some external links "unverified"?** Sites such as LinkedIn answer automated requests with 999 or 403 even when the link works. Those are not counted as broken.

**Is this affiliated with Google or any SEO tool?** No.

### Support

Open an issue on the Issues tab. Fixes for broken runs are a priority.

# Actor input Schema

## `startUrls` (type: `array`):

One website or start page per line (bare domains get https:// added). The Actor crawls each site from this page and its sitemap, staying on the same site (www. and non-www count as one site). At most 100 sites per run.

## `maxPagesPerSite` (type: `integer`):

Stop crawling a site after this many pages. Each audited page is one charged result. Up to 10,000.

## `checkExternalLinks` (type: `boolean`):

Also check links to other websites (one HEAD request each, after that site's robots.txt allows it). Servers that refuse bots (401, 403, 429, 999) are reported as unverified, not broken.

## `maxLinkChecks` (type: `integer`):

Links that are not crawled as pages (external links, files such as PDFs, and internal pages beyond the page limit) are checked with a lightweight request, up to this many per site. Internal links are checked first. Link checks are not charged.

## `useSitemap` (type: `boolean`):

Read the sitemaps listed in robots.txt (or /sitemap.xml) to find pages no link reaches, flag orphan pages, and check that sitemap URLs return 200 and are indexable.

## `includeSubdomains` (type: `boolean`):

Also crawl subdomains such as blog.example.com when auditing example.com.

## `excludeUrlPatterns` (type: `array`):

Skip pages whose URL contains any of these texts, for example /tag/ or ?sort=. Useful for shops with filter pages.

## `trackChanges` (type: `boolean`):

Remember each site's pages and issues in a named key-value store in your Apify account, and report new issues, fixed issues and alerts (for example a page that now returns 404, or became noindex) on the next run. Made for scheduled runs.

## `historyName` (type: `string`):

Keep separate histories for different schedules or clients. Letters, digits, '-' and '\_'.

## `maxConcurrency` (type: `integer`):

Requests in flight to the audited site at once. Requests start at most about 6 per second, and a Crawl-delay in robots.txt (up to 10 seconds) is honored.

## Actor input object example

```json
{
  "startUrls": [
    "https://example.com/"
  ],
  "maxPagesPerSite": 20,
  "checkExternalLinks": true,
  "maxLinkChecks": 50,
  "useSitemap": true,
  "includeSubdomains": false,
  "excludeUrlPatterns": [
    "/tag/",
    "?sort="
  ],
  "trackChanges": true,
  "historyName": "default",
  "maxConcurrency": 4
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `report` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/"
    ],
    "maxPagesPerSite": 20,
    "maxLinkChecks": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("stellalytics/seo-site-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://crawler-test.com/"],
    "maxPagesPerSite": 20,
    "maxLinkChecks": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("stellalytics/seo-site-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/"
  ],
  "maxPagesPerSite": 20,
  "maxLinkChecks": 50
}' |
apify call stellalytics/seo-site-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stellalytics/seo-site-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9CYx5h3ZOBEq3Mu85/builds/RfrFfuh3tsDk3bWDc/openapi.json