# Broken Link Checker: Find Dead Links, 404s & Redirects (`phaux/broken-link-checker`) Actor

Find broken links (404, 5xx, DNS, timeouts) and redirect chains on one website. Use for site audits after a redesign. Input: startUrls (one site or its sitemap), maxPages, maxUrlsChecked, checkExternalLinks. Output per URL: status code, category, redirect hops, source pages, anchor text.

- **URL**: https://apify.com/phaux/broken-link-checker.md
- **Developed by:** [Phaux](https://apify.com/phaux) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.20 / 1,000 checked urls

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker: Find Dead Links, 404s & Redirects

Check your website for **broken links** and **dead links**: a **broken link checker** for site owners, SEO audits and content migrations.

Give it your website (or its sitemap) and get a list of **every broken link, error page and redirect chain**, with the **page where each link appears and its anchor text**. Each unique URL is checked once, internal links always and external links when you switch them on.

### Quick start for AI agents and API users

- **What it does:** Find broken links (404, 5xx, DNS, timeouts) and redirect chains on one website.
- **When to use it:** For site audits after a redesign.
- **Example input** (the form's prefill; it finishes in under a minute):

```json
{
  "startUrls": [
    "https://crawlee.dev"
  ],
  "maxPages": 2,
  "maxUrlsChecked": 15,
  "requestIntervalSecs": 0.5
}
```

- **Example output:** one real dataset row from run `tI03Q30d6yOLLf0Rt` (2026-10-23), with long fields shortened and empty fields left out. Fields not shown are listed in the output table below.

```json
{
  "url": "https://crawlee.dev/",
  "finalUrl": "https://crawlee.dev/",
  "status": 200,
  "category": "ok",
  "redirectCount": 0,
  "contentType": "text/html",
  "responseTimeMs": 20,
  "method": "GET",
  "isInternal": true,
  "occurrences": 3,
  "foundOn": [
    {
      "pageUrl": "https://crawlee.dev/",
      "anchorText": "Crawlee",
      "tag": "a"
    },
    {
      "pageUrl": "https://crawlee.dev/js",
      "anchorText": "Crawlee",
      "tag": "a"
    }
  ],
  "firstFoundOn": "https://crawlee.dev/",
  "anchorText": "Crawlee",
  "depth": 0,
  "site": "crawlee.dev",
  "checkedAt": "2026-10-23T19:10:50Z"
}
```

- **Cost (pay per event, Free-plan prices):** Actor start $0.005; Checked URL $0.0002. Failed items are not charged.
- **From an AI agent:** with the Apify MCP server, call `call-actor` with actor `phaux/broken-link-checker`, or load it as its own tool with `https://mcp.apify.com?tools=phaux/broken-link-checker`. Read results with `get-dataset-items`.

### What does Broken Link Checker do?

- **Crawls your site** from a start URL, following internal links up to `maxDepth` and `maxPages`, or starts from the pages listed in a **sitemap** (`https://example.com/sitemap.xml`, sitemap indexes and `.xml.gz` included).
- **Checks every unique link once**: the final **HTTP status code**, every **redirect hop** (`301`, `302`, `307`, `308`) with its status, and network errors (DNS, connection refused, timeout, too many redirects).
- **Shows where each link is used**: up to 10 source pages per URL with anchor text (or image alt text), plus how many pages link to it.
- **Optional checks:** external links, and images, scripts, stylesheets and iframes (`checkResources`).
- **Sorts broken links first** and saves a `SUMMARY` with the broken links and their source pages.

### Why use it?

- **Fix 404s before your users and search engines find them**, after a redesign, migration or content cleanup.
- **Find redirect chains** (A → B → C) that waste crawl budget and slow pages down; `redirectChain` lists every hop.
- **Pay only per checked URL**; links that can't be checked (robots.txt, caps, time budget) are listed for free.
- **Safe defaults**: 50 pages and 1,000 checked URLs per run unless you raise them, and a time guard that saves results before the run timeout.

Use cases: SEO audits, website QA before a launch, link rot monitoring on blogs and documentation, migration checks (old URLs should redirect once, not three times), and agency site audits.

### How to use it

1. Put your site's start page (`https://example.com`) or sitemap URL in **Start URL or sitemap**. Empty input runs a small example on crawlee.dev.
2. Set **Max pages to crawl** and **Max URLs checked** (the second one caps your cost).
3. Switch on **Check external links** to also verify outbound links.
4. Run it and open the **All checked URLs** view; filter by `category` = `broken`. Export as CSV, Excel or JSON.

### Input

```json
{
  "startUrls": ["https://crawlee.dev"],
  "maxPages": 200,
  "maxUrlsChecked": 2000,
  "maxDepth": 3,
  "checkExternalLinks": true,
  "checkResources": false,
  "excludeUrlPatterns": ["/tag/", "\\?replytocom="],
  "requestIntervalSecs": 1
}
```

| Field | Type | Default | Description |
|---|---|---|---|
| `startUrls` | array | — | Your website's start page (e.g. https://example.com) or a sitemap URL (e.g. https://example.com/sitemap.xml). Several URLs are allowed if they are on the same website. Empty = a small example run on crawlee.dev. |
| `maxPages` | integer | `50` | How many internal HTML pages are opened to collect links (1–10,000). Links on those pages are all checked, so the number of checked URLs is usually larger. |
| `maxUrlsChecked` | integer | `1000` | Upper limit of checked URLs (= results you pay for). 0 = up to 100,000. URLs over the limit are listed in SKIPPED_LINKS for free. |
| `maxDepth` | integer | `3` | Link depth from the start page (0 = only check the links on the start pages). |
| `checkExternalLinks` | boolean | `false` | Also check links that point to other websites (one request each, never crawled further). |
| `checkResources` | boolean | `false` | Also check img, script, iframe, video/audio source and stylesheet/icon URLs, not only a/area links. |
| `useSitemap` | boolean | `false` | Add the pages from the site's sitemaps (found via robots.txt or /sitemap.xml) to the crawl, up to Max pages. Useful for finding orphan pages. |
| `includeUrlPatterns` | array | — | Regular expressions; when set, only internal pages whose URL matches one of them are crawled for links (links found on them are still all checked). Example: /blog/ |
| `excludeUrlPatterns` | array | — | Regular expressions; matching URLs are neither crawled nor checked. Example: /logout|?replytocom= |
| `requestIntervalSecs` | number | `1` | Politeness delay per host (0.5–30 s). A robots.txt Crawl-delay overrides it when longer. |
| `maxConcurrency` | integer | `5` | Requests in flight across different hosts (1–20). Requests to the same host are always one at a time. |
| `timeBudgetSecs` | integer | `0` | Stop crawling and checking after this many seconds and save the results so far (0 = only the run timeout guard, which always applies). |

### Output example

One dataset item per checked URL:

```json
{
  "url": "https://example.com/old-pricing",
  "category": "broken",
  "status": 404,
  "finalUrl": "https://example.com/pricing-2024",
  "redirectCount": 1,
  "redirectChain": [{"url": "https://example.com/old-pricing", "status": 301}],
  "errorCode": null,
  "isInternal": true,
  "firstFoundOn": "https://example.com/blog/launch",
  "anchorText": "see our pricing",
  "occurrences": 3,
  "foundOn": [{"pageUrl": "https://example.com/blog/launch", "anchorText": "see our pricing", "tag": "a"}],
  "contentType": "text/html",
  "responseTimeMs": 182,
  "method": "HEAD",
  "depth": 2,
  "site": "example.com",
  "checkedAt": "2026-10-20T21:40:00Z"
}
```

### Output fields

| Field | Type | Description |
|---|---|---|
| `url` | string | Checked URL (normalised, without #fragment). |
| `category` | string | broken (4xx/5xx or no response), redirect (ended OK after one or more redirects) or ok. |
| `status` | integer or null | Final HTTP status code; null when there was no HTTP response (DNS, connection or timeout error). |
| `finalUrl` | string or null | URL after following redirects. |
| `redirectCount` | integer | Number of redirect hops. |
| `redirectChain` | array | Each hop as {url, status}, in order, before finalUrl. |
| `errorCode` | string or null | dns_error, network_error, timeout, too_many_redirects, blocked_redirect or http_error when the check failed without a usable response. |
| `error` | string or null | Human-readable error. |
| `isInternal` | boolean | true when the URL is on the audited website (www. ignored). |
| `firstFoundOn` | string or null | First page where the link was found (null for start or sitemap URLs). |
| `anchorText` | string or null | Link text (or image alt) on firstFoundOn. |
| `occurrences` | integer | How many crawled pages link to this URL. |
| `foundOn` | array | Up to 10 source pages as {pageUrl, anchorText, tag}. |
| `contentType` | string or null | Content-Type of the final response. |
| `responseTimeMs` | integer or null | Time of the final request in milliseconds. |
| `method` | string | HEAD or GET (GET is used for crawled pages and when HEAD fails). |
| `depth` | integer | Link depth from the start page. |
| `site` | string | Audited website. |
| `checkedAt` | string | UTC time of the run. |

The key-value store also gets `SUMMARY` (counts and the broken links with their source pages), `SKIPPED_LINKS` (URLs not checked, with a code: `robots_disallowed`, `host_stopped`, `rate_limited`, `blocked_host`, `max_urls`, `time_budget`) and `RUN_STATS`.

### How much does it cost?

Pay per event, the same on every Apify plan. Apify platform usage is included.

| Event | Price |
|---|---|
| Actor start (once per run) | $0.005 |
| Checked URL (per result saved) | $0.0002 |

Example: 1,000 checked URLs cost $0.205; the small example run (17 URLs) costs about $0.0084. Skipped links and bad-input runs that stop before crawling are never charged per URL. Use **Max URLs checked** to cap any run.

### Use with AI agents, MCP and the API

- **MCP:** add `https://mcp.apify.com?tools=phaux/broken-link-checker` to an MCP client and ask, for example, "find broken links on example.com".
- **API:** `POST https://api.apify.com/v2/acts/phaux~broken-link-checker/run-sync-get-dataset-items` with `{"startUrls": ["https://example.com"], "maxUrlsChecked": 300}` returns the results directly.
- Every input and output field is described in the input and dataset schemas.

### Data source and compliance

- Meant for **your own websites** (or sites you are allowed to audit). Only public `http(s)` URLs are fetched; private and internal addresses are refused on every redirect hop.
- **robots.txt is honoured** (RFC 9309) for the user agent `PhauxLinkCheckBot`, including `Crawl-delay`. Disallowed links are skipped and listed in `SKIPPED_LINKS`.
- **Polite pacing:** requests to one host are sequential and at least `requestIntervalSecs` apart (default 1 s). After an **HTTP 429** a host is not contacted again in that run; if your own site answers 429, the crawl stops and the results so far are saved.
- It identifies itself honestly (`Mozilla/5.0 (compatible; PhauxLinkCheckBot/0.1; +https://apify.com/phaux/broken-link-checker)`), uses **no proxies and no IP rotation**, never logs in and doesn't get around bot protection.
- External links are checked with one HEAD request (GET when HEAD fails) and are never crawled further.
- Anchor texts are stored with email addresses and phone numbers removed.

### Limits

- One website per run (several start URLs are fine if they share the host; `www.` is ignored). Subdomains count as external.
- JavaScript-rendered links are not seen: the Actor reads the HTML the server sends.
- Up to 10,000 pages crawled and 100,000 URLs checked per run. At the default 1 request per second, 1,000 internal URLs take roughly 17 minutes.
- Some servers answer 403 or 429 to automated checks even when the page works in a browser; those show as broken or skipped with the real status code.

### FAQ

- **Why is a link "broken" when it works in my browser?** The server returned an error status to the checker (often 403 from bot protection, or 405 to HEAD; HEAD failures are retried with GET). The `status` field shows exactly what the server answered.
- **How do I see only broken links?** Filter the dataset by `category` = `broken`, or open the `SUMMARY` record.
- **What are redirect chains?** `redirectChain` lists every hop before `finalUrl`; `redirectCount` greater than 1 means a chain worth shortening.
- **Run stopped early?** `RUN_STATS.stoppedByTimeBudget` is true when the time guard was hit; raise the run timeout or lower `maxUrlsChecked`.
- **Something isn't working?** Open an issue on the Issues tab with the run ID.

# Actor input Schema

## `startUrls` (type: `array`):

Your website's start page (e.g. https://example.com) or a sitemap URL (e.g. https://example.com/sitemap.xml). Several URLs are allowed if they are on the same website. Empty = a small example run on crawlee.dev.

## `maxPages` (type: `integer`):

How many internal HTML pages are opened to collect links (1–10,000). Links on those pages are all checked, so the number of checked URLs is usually larger. Default: 50.

## `maxUrlsChecked` (type: `integer`):

Upper limit of checked URLs (= results you pay for). 0 = up to 100,000. URLs over the limit are listed in SKIPPED_LINKS for free. Default: 1000.

## `maxDepth` (type: `integer`):

Link depth from the start page (0 = only check the links on the start pages). Default: 3.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites (one request each, never crawled further). Default: false.

## `checkResources` (type: `boolean`):

Also check img, script, iframe, video/audio source and stylesheet/icon URLs, not only a/area links. Default: false.

## `useSitemap` (type: `boolean`):

Add the pages from the site's sitemaps (found via robots.txt or /sitemap.xml) to the crawl, up to Max pages. Useful for finding orphan pages. Default: false.

## `includeUrlPatterns` (type: `array`):

Regular expressions; when set, only internal pages whose URL matches one of them are crawled for links (links found on them are still all checked). Example: /blog/

## `excludeUrlPatterns` (type: `array`):

Regular expressions; matching URLs are neither crawled nor checked. Example: /logout|?replytocom=

## `requestIntervalSecs` (type: `number`):

Politeness delay per host (0.5–30 s). A robots.txt Crawl-delay overrides it when longer. Default: 1.

## `maxConcurrency` (type: `integer`):

Requests in flight across different hosts (1–20). Requests to the same host are always one at a time. Default: 5.

## `timeBudgetSecs` (type: `integer`):

Stop crawling and checking after this many seconds and save the results so far (0 = only the run timeout guard, which always applies). Default: 0.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawlee.dev"
  ],
  "maxPages": 2,
  "maxUrlsChecked": 15,
  "maxDepth": 3,
  "checkExternalLinks": false,
  "checkResources": false,
  "useSitemap": false,
  "requestIntervalSecs": 0.5,
  "maxConcurrency": 5,
  "timeBudgetSecs": 0
}
```

# Actor output Schema

## `results` (type: `string`):

All checked URLs, broken first (dataset, overview view).

## `summary` (type: `string`):

SUMMARY record: counts and the list of broken links with source pages.

## `skipped` (type: `string`):

Links not checked (robots.txt, HTTP 429, caps, time budget) with the reason.

## `runStats` (type: `string`):

RUN_STATS record.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawlee.dev"
    ],
    "maxPages": 2,
    "maxUrlsChecked": 15,
    "requestIntervalSecs": 0.5
};

// Run the Actor and wait for it to finish
const run = await client.actor("phaux/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://crawlee.dev"],
    "maxPages": 2,
    "maxUrlsChecked": 15,
    "requestIntervalSecs": 0.5,
}

# Run the Actor and wait for it to finish
run = client.actor("phaux/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawlee.dev"
  ],
  "maxPages": 2,
  "maxUrlsChecked": 15,
  "requestIntervalSecs": 0.5
}' |
apify call phaux/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,phaux/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/uJnzNbaxmMvuYZHpa/builds/i04pD4xsUSzTdkJfY/openapi.json