# Broken Link Checker: Find 404s, Dead Links, Redirects (`joesimo/broken-link-checker`) Actor

Crawl your website and check every internal and external link. Finds 404s, server errors, timeouts, DNS failures and redirect chains, with the page each link is on and its anchor text. Respects robots.txt. No login.

- **URL**: https://apify.com/joesimo/broken-link-checker.md
- **Developed by:** [Joe Simo](https://apify.com/joesimo) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.50 / 1,000 urls

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker: Find 404s, Dead Links, Redirects

Give it your website and it crawls your pages, collects every link, and checks each one. You get a clean list of broken links (404, 410, 5xx, timeouts, DNS and connection errors) with the **pages they appear on and their anchor text**, so you know exactly what to fix and where. Redirects get the full chain and final URL.

Empty input runs a short demo (10 pages of crawlee.dev, up to 100 checked URLs), so you can try it from the Store without filling in a form.

**$0.50 per 1,000 checked URLs** ($0.0005 per unique URL). Each URL is checked and charged once per run, however many of your pages link to it. Links skipped by robots.txt, your exclude patterns or your limits are free, and so are the site summary rows.

### What it does

- **Crawls your site** from the start URL (same host, www and non-www count as one), up to your page and depth limits. Internal links beyond the page limit are still status checked, just not crawled.
- **Checks every link once:** internal pages through the crawl itself, other links with one HEAD request (GET fallback when HEAD is refused or fails, so servers that mishandle HEAD do not show up as false 404s).
- **Follows redirects by hand** and records the chain, the final URL and redirect loops.
- **Tells you where each link lives:** up to 10 source pages per link (configurable) with anchor text, image alt text or aria-label, link type and a nofollow flag.
- **Fewer false alarms:** 401, 403, 429 and 999 answers (LinkedIn, some CDNs) usually mean the site refuses automated checks, so they are reported as `blocked_or_auth` and not as broken unless you turn on `treatBlockedAsBroken`.
- **Optional asset checks:** images, scripts, stylesheets, iframes and media files.
- **Several sites in one run,** each with a free `site_summary` row: pages crawled, links found, URLs checked, broken by issue type, redirects and links not checked.

### Use cases

- Find and fix 404s before they hurt SEO and user trust.
- Audit outbound links on blogs, docs and resource pages for dead or hijacked domains.
- Check a site after a migration or redesign for broken internal links and redirect chains.
- Schedule a weekly run and send new broken links to Slack, email or a sheet through Apify integrations.

### Input examples

| Goal | Input |
| --- | --- |
| Broken links on a site (first 100 pages) | `"startUrls": [{ "url": "https://example.com" }]` |
| Bigger site | `"maxPagesPerSite": 2000, "maxLinksPerSite": 20000` |
| Only the links on one page | `"startUrls": [{ "url": "https://example.com/resources" }], "maxDepth": 0` |
| Broken links plus redirects | `"outputMode": "issues"` |
| Internal links only | `"checkExternalLinks": false` |
| Skip tag and search pages | `"excludePatterns": ["/tag/", "?s="]` |
| Also check images and scripts | `"includeAssets": true` |
| Stop after 10 minutes with what was found | `"maxRunSecs": 600` |

### Output

One row per link (filtered by `outputMode`), sorted with broken links first, then one `site_summary` row per site.

```json
{
  "type": "link",
  "site": "https://www.python.org",
  "url": "http://wiki.python.org/moin/TkInter",
  "finalUrl": "https://wiki.python.org/moin/TkInter",
  "statusCode": 404,
  "broken": true,
  "issue": "not_found",
  "error": null,
  "internal": false,
  "linkTypes": ["a"],
  "redirectCount": 1,
  "redirectChain": [{ "url": "http://wiki.python.org/moin/TkInter", "statusCode": 301 }],
  "occurrences": 1,
  "foundOnCount": 1,
  "foundOn": [{ "pageUrl": "https://www.python.org/about/apps/", "anchorText": "Tk", "linkType": "a" }],
  "checkedVia": "check"
}
```

`issue` values: `not_found` (404, 410), `server_error` (5xx), `client_error` (other 4xx), `timeout`, `network_error` (DNS, refused, TLS, redirect loop), `blocked_or_auth` (401, 403, 429, 999), `redirect`, `redirect_unresolved`, `not_checked` (only in `all` mode), or `null` for a clean 2xx.

The `OUTPUT` record in the key-value store holds the run summary per site.

### Pricing

Pay per event: **$0.0005 per checked URL**. Examples:

| Site | Checked URLs | Cost |
| --- | --- | --- |
| Small site, 50 pages, 400 unique links | 400 | $0.20 |
| Blog, 300 pages, 2,500 unique links | 2,500 | $1.25 |
| Docs site, 2,000 pages, 15,000 unique links | 15,000 | $7.50 |

Set a maximum cost per run in the run options: the Actor stops cleanly at that amount and never goes over it. `maxLinksPerSite` (default 5,000) caps each site.

### Good to know

- **Long crawls end cleanly:** set `maxRunSecs` to cap the run time (0 means no limit). When the time is up, or 25 s before the run timeout in your run options, the Actor stops starting new checks, saves the rows and `site_summary` found so far and finishes as SUCCEEDED, with a warning in the log and `stopReason` in the summary. Links that were not checked in time are not charged.
- **Polite by design:** honest User-Agent (`BrokenLinkChecker/1.0` with this Store URL), a gap between requests to each host that widens after a 429, retries with backoff, no proxy, no browser, no login and no captcha solving. Private and local addresses are refused.
- **robots.txt** is respected for your own pages by default. Turn `respectRobotsTxt` off only for a site you own.
- **JavaScript-only links** (added by client-side scripts after load) are not seen, because pages are read as HTML without a browser. Most CMS, static and server-rendered sites put links in the HTML.
- **Some sites block all bots** and answer 403 or 999 to any checker. Those links show as `blocked_or_auth`, not broken, by default.
- Output is URLs, status codes and your own page text (anchor text). No personal data is collected.

### Related Actors

- [Sitemap Extractor](https://apify.com/joesimo/sitemap-extractor): every URL from a site's sitemaps, with optional status checks.
- [Tech Stack Detector](https://apify.com/joesimo/tech-stack-detector): which CMS, frameworks and tools a website uses.

Questions or a feature request? Open an issue on the Actor's Issues tab.

# Actor input Schema

## `startUrls` (type: `array`):

Home page or any page of each website. The crawler stays on the same host (www and non-www count as one) and checks every link it finds. Bare domains like example.com work too.

## `maxPagesPerSite` (type: `integer`):

How many of your own pages to open and read links from. Internal links beyond this limit are still status checked, just not crawled.

## `maxLinksPerSite` (type: `integer`):

Upper limit of unique URLs checked per site (crawled pages plus links). Each checked URL is one paid event, so this also caps cost per site.

## `maxDepth` (type: `integer`):

Clicks from the start page. 0 checks only the links on the start page.

## `outputMode` (type: `string`):

Broken only: links that failed (4xx, 5xx, timeouts, DNS errors). Issues: broken plus redirects and links that block automated checks. All: every link, including those not checked.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites (one HEAD request each, GET fallback).

## `includeAssets` (type: `boolean`):

Also check img, script, stylesheet, iframe and media URLs found on your pages.

## `treatBlockedAsBroken` (type: `boolean`):

Sites like LinkedIn answer automated checks with 403 or 999 even when the page works in a browser. Off by default, so these show as blocked_or_auth instead of broken.

## `excludePatterns` (type: `array`):

Links matching any of these are neither crawled nor checked. Plain text matches anywhere in the URL (case insensitive); /regex/ also works. Example: /tag/, ?replytocom=

## `respectRobotsTxt` (type: `boolean`):

Skip your own pages that robots.txt disallows for all user agents. Turn off only for a site you own.

## `maxFoundOnPerLink` (type: `integer`):

How many source pages (with anchor text) to list for each link.

## `concurrency` (type: `integer`):

Parallel requests per site. Each host also gets a short gap between requests.

## `timeoutSecs` (type: `integer`):

Links that do not answer in time are reported as timeout.

## `maxRunSecs` (type: `integer`):

Stop after this many seconds, save the links checked so far and finish cleanly (SUCCEEDED). Links not checked in time are not charged. 0 means no limit; the run still stops 25 s before the platform run timeout.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPagesPerSite": 10,
  "maxLinksPerSite": 150,
  "maxDepth": 10,
  "outputMode": "broken",
  "checkExternalLinks": true,
  "includeAssets": false,
  "treatBlockedAsBroken": false,
  "respectRobotsTxt": true,
  "maxFoundOnPerLink": 10,
  "concurrency": 5,
  "timeoutSecs": 20,
  "maxRunSecs": 60
}
```

# Actor output Schema

## `results` (type: `string`):

One row per checked link (filtered by the output mode) with status, issue, redirect chain and the pages it appears on, plus a free site_summary row per site.

## `runSummary` (type: `string`):

JSON summary per site: pages crawled, URLs checked, broken by issue, redirects, not checked and stop reason.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://crawlee.dev"
        }
    ],
    "maxPagesPerSite": 10,
    "maxLinksPerSite": 150,
    "maxRunSecs": 60
};

// Run the Actor and wait for it to finish
const run = await client.actor("joesimo/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://crawlee.dev" }],
    "maxPagesPerSite": 10,
    "maxLinksPerSite": 150,
    "maxRunSecs": 60,
}

# Run the Actor and wait for it to finish
run = client.actor("joesimo/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "maxPagesPerSite": 10,
  "maxLinksPerSite": 150,
  "maxRunSecs": 60
}' |
apify call joesimo/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,joesimo/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KQyKMAq3XK6MWLmyJ/builds/Ik0ykMd34GM8Dn6OI/openapi.json
