# Broken Link Checker - Sitemap & Site Audit (404s, Redirects) (`appetizing_horsefly/sitemap-broken-link-auditor`) Actor

Find broken links on any website from its sitemap or a capped crawl: 404s, 5xx, timeouts, DNS errors and redirect chains, with source page and anchor text. $0 when nothing is broken.

- **URL**: https://apify.com/appetizing_horsefly/sitemap-broken-link-auditor.md
- **Developed by:** [GM Labs](https://apify.com/appetizing_horsefly) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 broken links

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker: sitemap and site crawl audit for 404s, dead links and redirect chains

**Find every broken link on a website in one run.** Give it a homepage or a sitemap URL. It expands the sitemaps and sitemap indexes, crawls a capped number of public pages, checks each unique link (HEAD first, GET when HEAD isn't allowed), and returns **only the problems**: 404s and other 4xx/5xx errors, timeouts, DNS failures, SSL errors, redirect loops and redirect chains. Each row gives the page the link is on and its anchor text.

**Nothing found means nothing charged.** You pay only per broken link reported. Redirects, blocked links and OK links cost nothing.

Built for **SEO audits, site migrations, content teams, agencies running weekly client checks, and AI agents** that need a quick "is this site healthy?" answer.

### Why this one

- **One step from sitemap to audit**: no second Actor needed to get URLs. Sitemap indexes, `.xml.gz` and plain-text sitemaps all work, and sitemaps are found through `robots.txt`.
- **Sitemap entries are checked too**: a 404 listed in your sitemap wastes crawl budget.
- **Unique links, checked once**: a dead footer link on 300 pages counts as one row (with `foundOnPages` and up to 9 `otherSources`), and you're charged once.
- **Fewer false alarms**: links that answer 401/403/429 to automated checks (login walls, bot protection) are marked `blocked` and are free. Timeouts are retried once.
- **Polite and capped**: respects robots.txt, max 20 parallel requests, hard caps on pages (2000), links (10000) and run time.
- **Never fails on bad input**: an invalid URL or a site with no links ends SUCCEEDED with 0 rows, $0 and a clear status message.

Compared with desktop crawlers like Screaming Frog, it runs in the cloud on a schedule, needs no install, and is easy to use from n8n, Make, Zapier, the API or an AI agent.

### Try it free in 1 click

The input form is **prefilled with a small, working example**. Press **Start** without changing anything: the run audits 15 pages of crawlee.dev and lists broken, blocked and redirected links. Our test run of exactly this input finished in about 20 seconds with 74 results. At list price that is about $0.15 at most (you pay only per reported link), which fits inside the free monthly credit of Apify's free plan. Then swap in your own values.

Prefilled input:

```json
{
  "startUrl": "https://crawlee.dev",
  "maxPages": 15,
  "maxLinks": 300,
  "maxRunSecs": 180
}
```

### Sample output

Real rows from the prefilled run on 2026-10-14 (5 of 74 results shown; long fields trimmed). Download the full dataset as JSON, CSV or Excel.

| type | status | linkUrl | sourceUrl | foundOnPages |
|---|---|---|---|---|
| blocked | 403 | https://stackoverflow.com/questions/tagged/c… | https://crawlee.dev/ | 19 |
| redirect | 200 | https://www.crawlee.dev/blog/launching-crawl… | https://crawlee.dev/blog | 2 |
| redirect | 200 | https://www.semver.org/ | https://crawlee.dev/blog | 2 |
| broken | 404 | https://curl-cffi.readthedocs.io/en/latest/i… | https://crawlee.dev/blog/common-problems-in-… | 1 |
| blocked | 403 | https://www.upwork.com/freelancers/mantisus | https://crawlee.dev/blog/common-problems-in-… | 1 |

### Input (copy-paste)

```json
{
  "startUrl": "https://example.com",
  "followSitemaps": true,
  "crawlHtml": true,
  "checkExternal": true,
  "maxPages": 200,
  "maxLinks": 3000
}
```

Use `"startUrl": "https://example.com/sitemap.xml", "crawlHtml": false` to check only the URLs in a sitemap.

### API (copy-paste)

```bash
curl -X POST "https://api.apify.com/v2/acts/appetizing_horsefly~sitemap-broken-link-auditor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' -d '{"startUrl":"https://example.com","maxPages":100}'
```

### Output

```json
{
  "type": "broken",
  "sourceUrl": "https://example.com/blog/launch",
  "linkUrl": "https://example.com/docs/old-guide",
  "status": 404,
  "finalUrl": "https://example.com/docs/old-guide",
  "redirectChain": [],
  "anchorText": "read the setup guide",
  "error": "HTTP_404",
  "internal": true,
  "foundOnPages": 3,
  "checkMethod": "HEAD"
}
```

`type` is `broken`, `redirect` (with `redirectChain` like `["301 http://a", "200 https://a/"]`), `blocked` (401/403/429) or `ok` (only when "Only problems" is off). `error` is one of `HTTP_<code>`, `TIMEOUT`, `DNS_FAILURE`, `CONNECTION_REFUSED`, `CONNECTION_RESET`, `SSL_ERROR`, `REDIRECT_LOOP`, `TOO_MANY_REDIRECTS`. A run summary (pages crawled, links checked, counts, which cap stopped the run) is saved as `SUMMARY`.

### Pricing

Pay per event: **$0.002 per broken link reported** ($2 per 1,000). Redirect, blocked and OK rows are free, a clean site costs $0 in Actor fees, and setting a max charge limit stops the run when it's reached.

### Weekly monitoring

Schedule it weekly in Apify (Schedules) or use our n8n template, which sends a Slack and email digest of broken links that are new since the last run.

### Use it as an MCP tool (Claude, Cursor, ChatGPT, any AI agent)

This Actor works as a tool in the [Apify MCP server](https://mcp.apify.com), so an AI agent can call it directly and you pay per broken link, the same as a normal run. Add this server URL to any MCP client:

```
https://mcp.apify.com/?tools=appetizing_horsefly/sitemap-broken-link-auditor
```

Sign in with your Apify account (or pass your Apify API token as a Bearer header). Example request to the agent: "Check example.com for broken links and list the pages I need to fix."

### FAQ

- **JavaScript-only sites?** Links are read from the server HTML. Pages that build links only in the browser may show fewer links. The sitemap URLs are still checked.
- **Login pages?** Only public pages are crawled. Nothing is submitted and no cookies are used.
- **Why is a link "blocked" when it opens in my browser?** Some sites (LinkedIn, some CDNs) refuse automated requests. That's why these rows are free and kept apart from real broken links.

Made by GM Labs.

# Actor input Schema

## `startUrl` (type: `string`):

Homepage (sitemaps are found through robots.txt, /sitemap.xml and /sitemap_index.xml) or a direct sitemap URL (.xml, .xml.gz, sitemap index or plain-text). Example: https://example.com or https://example.com/sitemap.xml

## `followSitemaps` (type: `boolean`):

Read the site's sitemaps and sitemap indexes. Every URL listed there is checked (a 404 inside your sitemap is an SEO bug) and used as a page to crawl.

## `crawlHtml` (type: `boolean`):

Open public HTML pages (from the sitemap, then same-site links) and check every <a href> link on them, with anchor text. Turn off to check only the URLs listed in the sitemap.

## `checkExternal` (type: `boolean`):

Also check links that point to other domains. Turn off to audit internal links only.

## `maxPages` (type: `integer`):

Most HTML pages to open and take links from (and most sitemap URLs to check). Hard limit 2000.

## `maxLinks` (type: `integer`):

Most unique link URLs checked per run. Each link is checked once even if it appears on many pages. Hard limit 10000.

## `maxSitemaps` (type: `integer`):

Safety cap on nested sitemap files read.

## `onlyBroken` (type: `boolean`):

Output only broken links, redirect chains and blocked links. Turn off to also output every OK link (OK rows are free).

## `includeRedirects` (type: `boolean`):

Output links that redirect (301/302/307/308) with the full chain and final URL, so you can update them to the final address. Redirect rows are free.

## `includeBlocked` (type: `boolean`):

Output links where the server refused an automated check (login walls, bot protection, rate limits). These are often fine in a browser, so they are free and marked type=blocked.

## `countBlockedAsBroken` (type: `boolean`):

Count refused links as broken (charged) instead of 'blocked'. Leave off unless those links should really be public.

## `respectRobots` (type: `boolean`):

Skip pages and links that the site's robots.txt disallows for crawlers.

## `requestTimeoutSecs` (type: `integer`):

Seconds before a link counts as a TIMEOUT. Timeouts are retried once.

## `concurrency` (type: `integer`):

Parallel requests. Hard limit 20, to stay polite to sites.

## `maxRunSecs` (type: `integer`):

Stop crawling and checking after this many seconds and output what was found.

## Actor input object example

```json
{
  "startUrl": "https://crawlee.dev",
  "followSitemaps": true,
  "crawlHtml": true,
  "checkExternal": true,
  "maxPages": 15,
  "maxLinks": 300,
  "maxSitemaps": 50,
  "onlyBroken": true,
  "includeRedirects": true,
  "includeBlocked": true,
  "countBlockedAsBroken": false,
  "respectRobots": true,
  "requestTimeoutSecs": 15,
  "concurrency": 10,
  "maxRunSecs": 180
}
```

# Actor output Schema

## `results` (type: `string`):

All rows in the default dataset

## `summary` (type: `string`):

Pages crawled, links checked, broken/redirect/blocked counts, caps hit

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://crawlee.dev",
    "maxPages": 15,
    "maxLinks": 300,
    "maxRunSecs": 180
};

// Run the Actor and wait for it to finish
const run = await client.actor("appetizing_horsefly/sitemap-broken-link-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrl": "https://crawlee.dev",
    "maxPages": 15,
    "maxLinks": 300,
    "maxRunSecs": 180,
}

# Run the Actor and wait for it to finish
run = client.actor("appetizing_horsefly/sitemap-broken-link-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://crawlee.dev",
  "maxPages": 15,
  "maxLinks": 300,
  "maxRunSecs": 180
}' |
apify call appetizing_horsefly/sitemap-broken-link-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,appetizing_horsefly/sitemap-broken-link-auditor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/lYNCFWzZgHF3pf5Sw/builds/sq0yKYASffxJyxjr8/openapi.json