# Broken Link Checker – Find 404s on Any Website (`innovative_cookie/broken-link-checker`) Actor

Crawl a website and find broken links: 404s, 5xx errors, dead external links and timeouts, with the source page and anchor text. Fast, no browser.

- **URL**: https://apify.com/innovative_cookie/broken-link-checker.md
- **Developed by:** [Innovative Cookie](https://apify.com/innovative_cookie) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Broken Link Checker do?

**Broken Link Checker** crawls a website and **finds every broken link**: 404 pages, 5xx server errors, dead external links, DNS failures and timeouts. For each broken link you get the **page it was found on**, the **anchor text**, the HTTP status code and the reason, so you can fix it in minutes.

- 🔗 Checks **internal and external links** (and optionally images, scripts, stylesheets and iframes).
- 🧭 Crawls the site **breadth-first** from one or more start URLs, with page and depth limits.
- ↪️ Follows **redirect chains** (301, 302, 307, 308) and reports where each link ends up.
- 🤖 **Polite and transparent**: respects robots.txt and `Crawl-delay`, identifies itself, never bypasses captchas or bot protection.
- ⚡ Plain HTTP requests (HEAD first, GET to confirm), no browser and no proxy, so runs are fast and cheap.

### Why use Broken Link Checker?

- 🔎 **SEO audits.** Broken links waste crawl budget and leak link equity. Find them before Google does.
- 🚚 **Site migrations and redesigns.** Run it after a launch to catch links to pages that moved or disappeared.
- 📝 **Content maintenance.** Old blog posts often link to external pages that no longer exist ("link rot").
- 🛒 **E-commerce.** Find product and category links that return 404 after stock changes.
- ⏰ **Monitoring.** Schedule a weekly run and get alerted (email, Slack, webhook) when new broken links appear.

### How to find broken links on a website

1. Click **Try for free**.
2. In **Start URLs**, enter your homepage (`example.com`) or a section (`https://example.com/blog/`).
3. Set **Max pages to crawl** and **Max crawl depth** (start small, for example 50 pages, depth 2).
4. Click **Start**. A small site takes about a minute.
5. Download the broken links as **CSV, Excel or JSON**, or get them via the API.

### Input

See the **Input** tab for every option. Example:

```json
{
    "startUrls": ["https://crawler-test.com/links/broken_links_internal", "https://crawler-test.com/links/broken_links_external"],
    "maxPagesToCrawl": 2,
    "maxDepth": 0,
    "checkExternalLinks": true,
    "reportMode": "broken"
}
```

`reportMode` can be `broken` (default), `problems` (broken, redirected, blocked and skipped links) or `all` (every link with its status).

### Output

One row per broken link occurrence (a link that is broken on 30 pages is listed with each source page, up to a cap you choose). Example:

```json
[
    {
        "url": "https://crawler-test.com/links/not_found/foo1",
        "status": "broken",
        "statusCode": 404,
        "error": "HTTP 404",
        "sourceUrl": "https://crawler-test.com/links/broken_links_internal",
        "anchorText": "Broken Link 1",
        "linkType": "internal",
        "element": "a",
        "nofollow": false,
        "finalUrl": null,
        "redirectCount": 0,
        "firstRedirectStatus": null,
        "site": "https://crawler-test.com/links/broken_links_internal"
    }
]
```

| Field | Description |
|---|---|
| `url` | Link target (fragment removed) |
| `status` | `broken`, `redirect`, `blocked`, `skipped` or `ok` |
| `statusCode`, `error` | HTTP status and reason (`HTTP 404`, `Server error HTTP 500`, `Host unreachable (ENOTFOUND)`, timeout…) |
| `sourceUrl`, `anchorText` | Where the link was found and its text |
| `linkType`, `element`, `nofollow` | internal / external, HTML element, rel=nofollow |
| `finalUrl`, `redirectCount`, `firstRedirectStatus` | Redirect information |

The **`SUMMARY`** record in the key-value store gives totals per start URL: pages crawled, links found, broken, blocked, skipped and redirected links.

#### What "blocked" and "skipped" mean

- **blocked**: the server answered 401, 403 or 429, or showed a captcha / bot challenge, or its robots.txt could not be read (5xx). The link may work in a browser, so it is **not counted as broken**, and the Actor never tries to get around the protection.
- **skipped**: the link is disallowed by the target site's robots.txt, or points to a private address. It is not requested.

### How much does it cost to check broken links?

The Actor is **free to use** for now: there is no Actor fee, you only pay the Apify platform usage of your runs, which stays small because it uses plain HTTP requests. The Apify Free plan includes $5 of monthly credit. Keep runs short with **Max pages to crawl** and **Max links to check**.

### Integrations

Use the results anywhere with the Apify **API**, **webhooks**, **Make**, **Zapier**, **n8n**, **Google Sheets** or **Slack**, or call it from an AI agent through the Apify **MCP server**. Schedule a weekly run to monitor your site.

### FAQ

#### How do I find 404 errors on my site?

Enter your homepage, set a page limit and start the run. Every link returning 404 (or another error) is listed with the page that contains it.

#### Does it check external links?

Yes, by default. Each external URL is requested once (HEAD, then GET if the server mishandles HEAD), after reading that site's robots.txt. Turn off **Check external links** to only check your own pages.

#### Does it respect robots.txt?

Yes. It reads robots.txt before crawling or checking any host, re-checks it at every redirect, honours `Crawl-delay` (capped at 10 s by default) and spaces requests to the same host (250 ms by default). If robots.txt answers with a captcha, 401, 403, 429 or 5xx, that host is treated as "no bots allowed" and is not requested.

#### Does it render JavaScript?

No. It reads the HTML returned by the server, which covers most websites. Links that only exist after JavaScript runs are not seen.

#### Is it legal?

It only requests public pages and reads links, and collects **no personal data**. Only check websites you own or have the right to audit, and follow their terms of use.

#### Something does not work?

Open an issue in the **Issues** tab with the start URL and the `SUMMARY` record.

### Related actors

More Actors by the same author ([innovative_cookie on Apify](https://apify.com/innovative_cookie)):

- [Sitemap URL Extractor](https://apify.com/innovative_cookie/sitemap-extractor): get every URL of a site from its XML sitemaps.
- [PageSpeed Insights Bulk Checker](https://apify.com/innovative_cookie/pagespeed-bulk): Core Web Vitals and Lighthouse scores for many URLs.
- [Website Image Extractor](https://apify.com/innovative_cookie/website-image-extractor): list every image of a page with size, format and missing alt text.
- [Dataset Merge, Dedup & Transform](https://apify.com/innovative_cookie/dataset-merge-dedup): merge and clean the results of several runs.

# Actor input Schema

## `startUrls` (type: `array`):

One website or page per line (example.com or https://example.com/blog/). The crawler stays on the same host (www. is ignored) and checks every link it finds.

## `maxPagesToCrawl` (type: `integer`):

Stop crawling after this many HTML pages (all sites together). Links found on these pages are all checked, even if their target page is not crawled.

## `maxDepth` (type: `integer`):

0 = check only the links on the start pages. 1 = also crawl the pages they link to, and so on.

## `maxLinksToCheck` (type: `integer`):

Safety cap on the number of unique link URLs checked in one run.

## `includePatterns` (type: `array`):

Crawl only internal pages whose URL matches one of these regular expressions (links on crawled pages are still all checked). Example: /blog/

## `excludePatterns` (type: `array`):

Do not crawl internal pages matching these regular expressions. Example: /tag/|?page=

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites (one HEAD request each, robots.txt respected).

## `checkResources` (type: `boolean`):

Besides <a href> links, also check <img src>, <script src>, stylesheets and iframes.

## `includeSubdomains` (type: `boolean`):

Treat subdomains of the start host (blog.example.com) as internal pages and crawl them too.

## `reportMode` (type: `string`):

broken = only broken links (default). problems = broken, redirected, blocked (401/403/429) and skipped links. all = every link with its status.

## `maxOccurrencesPerLink` (type: `integer`):

A broken link that appears on many pages (menu, footer) is listed at most this many times.

## `maxConcurrency` (type: `integer`):

Pages crawled in parallel. Requests to the same host are always spaced out.

## `requestDelayMs` (type: `integer`):

Minimum pause between two requests to the same host. A larger robots.txt Crawl-delay is honoured.

## `maxCrawlDelaySecs` (type: `integer`):

Upper bound for the robots.txt Crawl-delay directive.

## `requestTimeoutSecs` (type: `integer`):

A link that does not answer within this time is reported as broken (timeout).

## Actor input object example

```json
{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal",
    "https://crawler-test.com/links/broken_links_external"
  ],
  "maxPagesToCrawl": 2,
  "maxDepth": 2,
  "maxLinksToCheck": 2000,
  "includePatterns": [],
  "excludePatterns": [],
  "checkExternalLinks": true,
  "checkResources": false,
  "includeSubdomains": false,
  "reportMode": "broken",
  "maxOccurrencesPerLink": 20,
  "maxConcurrency": 5,
  "requestDelayMs": 250,
  "maxCrawlDelaySecs": 10,
  "requestTimeoutSecs": 20
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset with one row per broken link occurrence

## `summary` (type: `string`):

SUMMARY record with totals per site

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/links/broken_links_internal",
        "https://crawler-test.com/links/broken_links_external"
    ],
    "maxPagesToCrawl": 2,
    "maxDepth": 0
};

// Run the Actor and wait for it to finish
const run = await client.actor("innovative_cookie/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        "https://crawler-test.com/links/broken_links_internal",
        "https://crawler-test.com/links/broken_links_external",
    ],
    "maxPagesToCrawl": 2,
    "maxDepth": 0,
}

# Run the Actor and wait for it to finish
run = client.actor("innovative_cookie/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal",
    "https://crawler-test.com/links/broken_links_external"
  ],
  "maxPagesToCrawl": 2,
  "maxDepth": 0
}' |
apify call innovative_cookie/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,innovative_cookie/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/nIydwKsAbayopud6v/builds/aM1NLeEcFmrHqM5FE/openapi.json