# Technical SEO Audit & Monitor: Broken Links, Redirects (`succinct_glider/technical-seo-audit`) Actor

Crawl a website and audit every page for technical SEO issues: broken links, redirects, noindex, canonical, titles, meta descriptions, H1, hreflang, structured data and sitemap coverage. Schedule it to get only the pages that changed or broke since the last audit.

- **URL**: https://apify.com/succinct\_glider/technical-seo-audit.md
- **Developed by:** [Alexandre Bobichon](https://apify.com/succinct_glider) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$5.00 / 1,000 pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**Audit a website for technical SEO issues** and get one row per page: **broken links, redirects and
redirect chains, noindex, canonical problems, missing or duplicate titles and meta descriptions, H1,
hreflang, structured data and sitemap coverage**, with a 0-100 score per page.

Run it on a schedule with **change tracking** and it becomes an **SEO monitor**: each run tells you which
pages broke, got de-indexed, lost their title or left the sitemap since the previous audit — and only
those pages are saved and charged.

Try it with the prefilled input: 20 pages of python.org, results in about half a minute.

### What does this technical SEO audit tool do?

It crawls a website the way a search engine does, starting from the URL you give it and following internal
links, then reads the sitemap to find pages that no link points to. Every URL is requested **without
following redirects**, so a `301` is reported on the URL that answers it, not hidden behind the final page.

Each page gets:

- its **HTTP status**, redirect target, response time and HTML size;
- its **indexability**, and the reason when it is not indexable (`noindex`, canonical to another page,
  redirect, error);
- the **SEO tags** that matter: title, meta description, H1, canonical, robots meta and `X-Robots-Tag`,
  `lang`, hreflang alternates, JSON-LD types;
- its **links**: internal and external counts, the **broken internal pages it links to**, the links that
  redirect, and how many crawled pages link to it;
- a list of **issues** with a severity (error, warning, notice) and a plain-English message, and a **score**.

Site-wide checks run once the crawl is done: **duplicate titles and descriptions**, **pages missing from the
sitemap**, **sitemap entries that are broken or not indexable**, and **orphan pages** (in the sitemap, linked
from nowhere).

It is plain HTTP, no browser: fast and cheap. The trade-off is stated below in the FAQ.

### Why audit and monitor technical SEO?

- **Catch regressions before rankings drop.** A deploy that adds `noindex` to a template, a CMS migration that
  breaks 200 URLs, a canonical that suddenly points to the homepage: schedule the Actor daily and get only
  those pages.
- **Agency and freelance reporting.** Audit every client site on a schedule, export to Google Sheets or
  Excel, and show the issues fixed since last month (`resolvedIssues`).
- **Pre- and post-migration checks.** Compare the site before and after a redesign or a domain move.
- **Content hygiene at scale.** Find every page with a missing meta description or a duplicate title in one
  run, instead of clicking through a CMS.
- **Feed your own tools.** A clean JSON row per URL, ready for a dashboard, a data warehouse or an AI agent.

### How to run a technical SEO audit

1. Enter the website in **Websites to audit** (`https://www.example.com/` or just `example.com`). Several
   websites can go in one run.
2. Set **Max pages per site**. Start with the default to check the output, then raise it above the size of
   the site.
3. Click **Start**, then open the **Issues** view of the Output tab.
4. To monitor the site, open **Schedules**, run the Actor daily or weekly, and switch on **Save only changes**.

#### What happens during a run

1. **robots.txt is read first.** URLs it disallows are never requested, and never charged.
2. **The start page is fetched**, then every internal link is followed, page by page, up to your limit.
   Redirects are recorded, not followed blindly: each URL keeps its own status code.
3. **The sitemap is read** and its URLs that no link points to are audited last, which is how orphan pages
   are found.
4. **Site-wide checks run once the crawl is done** — duplicate titles and descriptions, broken internal
   links, sitemap coverage — because each of them needs every page at once.
5. **Results are saved**, one row per URL, and compared with the previous audit when tracking is on.

Files that are not web pages (PDF, images, archives) are never requested, and a response that is not HTML is
recorded from its headers alone, so you are not charged for downloading them.

### Input

| Field                  | Default                     | What it does                                                       |
| ---------------------- | --------------------------- | ------------------------------------------------------------------ |
| `startUrls`            | –                           | Websites to audit, one per line (URL or domain).                   |
| `maxPagesPerSite`      | 20                          | Pages audited per website (up to 2,000).                           |
| `maxItems`             | 100                         | Pages audited in the whole run.                                    |
| `useSitemap`           | `true`                      | Read the sitemap to find orphan pages and check sitemap coverage.  |
| `includeSubdomains`    | `false`                     | Also crawl `blog.example.com` when auditing `example.com`.         |
| `excludeUrlPatterns`   | –                           | Skip URLs containing any of these texts (`/tag/`, `?replytocom=`). |
| `trackChanges`         | `false`                     | Mark each page new, changed, unchanged or removed.                 |
| `onlyChanges`          | `false`                     | Save only new, changed and removed pages (implies `trackChanges`). |
| `stateStoreName`       | `technical-seo-audit-state` | Named key-value store holding the previous audit.                  |
| `maxConcurrency`       | 4                           | Pages requested in parallel.                                       |
| `maxRequestsPerMinute` | 120                         | Crawl rate limit.                                                  |
| `proxyConfiguration`   | no proxy                    | Only for sites that block datacenter IPs.                          |

```json
{
    "startUrls": ["https://www.example.com/"],
    "maxPagesPerSite": 500,
    "maxItems": 500,
    "onlyChanges": true,
    "excludeUrlPatterns": ["/tag/", "?replytocom="]
}
```

### Output

One item per audited URL. You can download the dataset as JSON, CSV, Excel, XML or HTML, or read it through
the API. A shortened example:

```json
{
    "url": "https://www.example.com/pricing/",
    "siteUrl": "https://www.example.com/",
    "statusCode": 200,
    "redirectUrl": null,
    "responseTimeMs": 182,
    "depth": 1,
    "inlinkCount": 14,
    "inSitemap": true,
    "indexable": true,
    "nonIndexableReason": null,
    "title": "Pricing | Example",
    "titleLength": 17,
    "metaDescription": null,
    "h1": "Simple pricing",
    "h1Count": 1,
    "canonicalUrl": "https://www.example.com/pricing/",
    "noindex": false,
    "hreflang": [{ "lang": "fr", "url": "https://www.example.com/fr/tarifs/" }],
    "jsonLdTypes": ["Product"],
    "internalLinkCount": 48,
    "externalLinkCount": 3,
    "brokenLinks": [{ "url": "https://www.example.com/old-plan/", "statusCode": 404 }],
    "redirectingLinkCount": 2,
    "issues": [
        { "code": "broken_links", "severity": "error", "message": "The page links to 1 broken page(s) of the site." },
        { "code": "links_to_redirect", "severity": "warning", "message": "The page links to 2 URL(s) that redirect." },
        { "code": "missing_meta_description", "severity": "warning", "message": "The page has no meta description." },
        { "code": "title_too_short", "severity": "notice", "message": "The title is 17 characters (under 30)." }
    ],
    "errorCount": 1,
    "warningCount": 2,
    "noticeCount": 1,
    "score": 69,
    "changeStatus": "changed",
    "changedFields": ["metaDescription"],
    "previousValues": { "metaDescription": "Plans for every team." },
    "newIssues": ["missing_meta_description"],
    "resolvedIssues": [],
    "previousScrapedAt": "2026-09-16T06:00:04.112Z",
    "scrapedAt": "2026-09-17T06:00:03.871Z"
}
```

#### Fields you get for every page

| Field                                                          | What it tells you                                         |
| -------------------------------------------------------------- | --------------------------------------------------------- |
| `statusCode`, `redirectUrl`, `fetchError`                      | What the URL answers, and where it redirects              |
| `indexable`, `nonIndexableReason`                              | Whether a search engine can index it, and why not         |
| `title`, `titleLength`, `metaDescription`, `h1`, `h1Count`     | The tags that show up in search results                   |
| `canonicalUrl`, `metaRobots`, `xRobotsTag`, `noindex`          | Indexing directives, from the page and from the headers   |
| `lang`, `hreflang`, `jsonLdTypes`                              | Language targeting and structured data                    |
| `brokenLinks`, `redirectingLinkCount`, `internalLinkCount`     | Which links on the page are broken or redirect            |
| `inlinkCount`, `depth`, `inSitemap`                            | How the page is reached, and whether the sitemap lists it |
| `responseTimeMs`, `htmlSizeBytes`, `wordCount`                 | Speed and weight, and how much text the page has          |
| `issues`, `errorCount`, `warningCount`, `score`                | Every problem found, with a severity and a plain message  |
| `changeStatus`, `changedFields`, `newIssues`, `previousValues` | What changed since the previous audit                     |

#### The run summary

Besides the per-page rows, each run writes a `SUMMARY` record to the key-value store, with one entry per
website: pages audited, average score, issue counts, the most common issue codes, whether the crawl covered
the whole site, and why a website could not be audited when that happens.

```json
{
    "sitesRequested": 1,
    "pagesSaved": 500,
    "sites": [
        {
            "siteUrl": "https://www.example.com/",
            "status": "ok",
            "pagesAudited": 500,
            "complete": true,
            "averageScore": 88,
            "issueCounts": { "error": 4, "warning": 121, "notice": 260 },
            "topIssues": [
                { "code": "meta_description_too_short", "pages": 96 },
                { "code": "duplicate_title", "pages": 41 },
                { "code": "broken_links", "pages": 4 }
            ]
        }
    ],
    "budgetReached": false
}
```

#### Issues checked

| Severity | Codes                                                                                                                                                                                                                                                                                                                       |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Error    | `http_4xx`, `http_5xx`, `fetch_failed`, `broken_links`, `redirect_to_error`, `redirect_without_location`, `missing_title`, `canonical_to_error`                                                                                                                                                                             |
| Warning  | `redirect_chain`, `links_to_redirect`, `missing_meta_description`, `missing_h1`, `duplicate_title`, `duplicate_meta_description`, `canonical_to_redirect`, `sitemap_non_indexable`, `orphan_page`, `invalid_json_ld`, `invalid_hreflang`, `hreflang_on_non_indexable`, `mixed_content`, `missing_viewport`, `slow_response` |
| Notice   | `redirect`, `title_too_long` (> 60), `title_too_short` (< 30), `meta_description_too_long` (> 160), `meta_description_too_short` (< 70), `multiple_h1`, `missing_canonical`, `canonicalized`, `noindex`, `nofollow_page`, `not_in_sitemap`, `missing_lang`, `low_word_count` (< 100), `large_html` (> 2 MB)                 |

The score starts at 100 and loses 20 points per error, 5 per warning and 1 per notice, down to 0. Title,
description and heading checks only apply to indexable pages.

### Monitoring: tracking changes between audits

With `trackChanges` (or `onlyChanges`), each page is compared with the previous run of the same website:

- `new` — not seen in the previous audit;
- `changed` — status, redirect, indexability, `noindex`, canonical, title, meta description, H1 or sitemap
  presence changed, or issues appeared or were fixed (`changedFields`, `previousValues`, `newIssues`,
  `resolvedIssues`);
- `unchanged` — nothing moved (not saved with `onlyChanges`);
- `removed` — audited last time, no longer reached by the crawl (unlinked, deleted, or now excluded).

Removed pages are only reported when the crawl covered the whole site: if the page limit cut it short, the
Actor says so in the summary and never reports pages as removed. **Set `maxPagesPerSite` above the size of
the site for monitoring.** The first tracked run marks every page as new.

### Use it from the API, integrations or an AI agent

```bash
curl -X POST "https://api.apify.com/v2/acts/succinct_glider~technical-seo-audit/run-sync-get-dataset-items?token=<YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["example.com"], "maxPagesPerSite": 50}'
```

Send the results to Google Sheets, Slack, a webhook, Make, Zapier or n8n with Apify integrations, or let an
AI agent call the Actor through the [Apify MCP server](https://mcp.apify.com).

### Pricing

**$5 per 1,000 audited pages ($0.005 per page). No start fee.**

- Only saved pages are charged. With `onlyChanges`, unchanged pages are free, so monitoring a stable
  500-page site costs a few cents per run.
- A site that cannot be audited (blocked, down, disallowed by robots.txt) is reported in the summary and
  costs nothing.
- Set a **maximum cost per run** in the run options: the Actor stops cleanly when it is reached.

### Tips

- **Exclude the noise** with `excludeUrlPatterns`: tag archives, search results, calendar pages, `?sort=`
  parameters. Excluded URLs are never requested.
- **Keep the rate gentle** on small servers: `maxConcurrency` 2 and `maxRequestsPerMinute` 60.
- **Use one `stateStoreName` per monitoring job** if you run several schedules on the same website with
  different settings.
- **Start from a section** (`https://example.com/blog/`): the crawl starts there but still follows every link
  of the site.

### FAQ

**Does it render JavaScript?**
No. It reads the HTML the server sends, like most SEO crawlers in their default mode. Pages whose content,
titles or links are injected by JavaScript after load will look emptier than they are in a browser.
Single-page apps without server-side rendering are not a good fit.

**How large a website can it audit?**
Up to 2,000 pages per website per run, which keeps a run within the default 1 GB of memory. For larger
websites, audit sections separately with `excludeUrlPatterns`, and open an issue if you need more.

**Does it check external links?**
Not yet: it counts them, but only internal pages are requested. External link checking is planned.

**Can it audit a staging site behind a login or a password?**
No. It only reads pages a visitor can open without signing in, and it never fills in credentials. Put the
staging site behind an IP allowlist rather than a password if you want to audit it.

**How long does a run take, and what does it cost in compute?**
About 120 pages per minute with the default rate limit, so a 1,000-page audit takes around 10 minutes. On a
1,000-page run measured on the platform, compute was about $0.08 — and platform usage is included in the
price per page, so it is not billed to you on top.

**Does it respect robots.txt?**
Always, and it cannot be turned off. Pages that robots.txt disallows are skipped and not charged. The crawler
identifies itself as `TechnicalSeoAuditBot` in its User-Agent, and it only follows links within the website
you entered.

**Is it legal to audit a website?**
Audit **your own websites or your clients'** with their permission. The Actor only requests public pages,
respects robots.txt, crawls at a modest rate, and does not store page content — only SEO tags, statuses and
counts. You are responsible for the websites you choose to audit and for complying with their terms.

**A website came back with no results.**
The run summary tells you why: `unreachable` (the start page answered an error or blocked the request — try
a proxy), `disallowed` (robots.txt), or `redirected_away` (the URL redirects to another domain — audit that
one instead). None of these fail the run, and none is charged.

**Why is my page marked as not in the sitemap when it is?**
URLs are compared exactly: `https://example.com/page` and `https://example.com/page/` are different URLs for
a search engine. If the sitemap is very large (over 50,000 URLs), sitemap coverage is not checked at all
rather than reported wrongly.

### Support

Found a false positive, or a check you would like to see? Open an issue on the **Issues** tab with the page
URL and the issue code, and I will look at it.

# Changelog

This Actor's version history is a separate document: https://apify.com/succinct\_glider/technical-seo-audit/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

One website per line, as a URL or a domain (https://example.com or example.com). The crawl starts there and stays on that host (www. and the bare domain count as the same site).

## `maxPagesPerSite` (type: `integer`):

Pages audited on each site, up to 2,000. Each audited page is one result. For change tracking, set it above the size of the site so every run audits the same pages.

## `maxItems` (type: `integer`):

Hard limit on the number of pages audited in this run, across all sites.

## `useSitemap` (type: `boolean`):

Reads the sitemaps declared in robots.txt (or /sitemap.xml) to find pages that no link points to, and flags sitemap entries that are broken, redirected or not indexable.

## `includeSubdomains` (type: `boolean`):

Also crawl subdomains of the site (blog.example.com when auditing example.com).

## `excludeUrlPatterns` (type: `array`):

Skip URLs that contain one of these texts, e.g. /tag/, ?replytocom=, /wp-admin/. Skipped URLs are not audited and not charged.

## `trackChanges` (type: `boolean`):

Compares each page with the previous run and marks it new, changed, unchanged or removed, with the issues that appeared or were fixed. Made for scheduled runs.

## `onlyChanges` (type: `boolean`):

Saves only new, changed and removed pages (turns change tracking on). Unchanged pages are not saved and not charged. The first run saves every page, as all pages are new.

## `stateStoreName` (type: `string`):

Named key-value store holding the previous audit of each site. Use a different name for each independent monitoring job.

## `maxConcurrency` (type: `integer`):

Pages requested at the same time. Kept low by default to stay gentle on the audited site.

## `maxRequestsPerMinute` (type: `integer`):

Upper bound on the crawl rate, per run.

## `proxyConfiguration` (type: `object`):

Not needed for most sites. Use a proxy only if the audited site blocks the Apify datacenter IPs.

## Actor input object example

```json
{
  "startUrls": [
    "https://www.python.org/"
  ],
  "maxPagesPerSite": 20,
  "maxItems": 100,
  "useSitemap": true,
  "includeSubdomains": false,
  "trackChanges": false,
  "onlyChanges": false,
  "stateStoreName": "technical-seo-audit-state",
  "maxConcurrency": 4,
  "maxRequestsPerMinute": 120,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://www.python.org/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("succinct_glider/technical-seo-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://www.python.org/"] }

# Run the Actor and wait for it to finish
run = client.actor("succinct_glider/technical-seo-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://www.python.org/"
  ]
}' |
apify call succinct_glider/technical-seo-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,succinct_glider/technical-seo-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/uIbk0ZfIY70JLRtr1/builds/hsNOMdbIN5SKLinwA/openapi.json