# Broken Link Checker & Redirect Chain Finder (`kelp_forest_labs/broken-link-checker`) Actor

Crawl a website and check every internal and external link: status code, full redirect chain, broken links (404, timeouts, DNS errors), and the pages and anchor text where each link appears. Honours robots.txt and Crawl-delay. Skipped links are free.

- **URL**: https://apify.com/kelp_forest_labs/broken-link-checker.md
- **Developed by:** [kelp forest](https://apify.com/kelp_forest_labs) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.80 / 1,000 link checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Give it your start URL(s). It crawls **your own site**, collects every link, and checks each unique link **once**:
HTTP status, the **full redirect chain**, the final URL, and **which pages link to it** (with anchor text), so you
can fix broken links and redirect hops quickly.

### Use cases

- Monthly SEO link audits for clients.
- Find redirect chains after a site migration.
- Catch dead outbound links in docs and blogs.
- Pre-launch QA of a new site.

### What it does

- Breadth-first crawl of each start URL's host, limited by `maxDepth`, `maxPages` and `maxLinks`. Every start URL
  crawls its own scope to `maxDepth`, even where it overlaps another start URL; a link already checked is not
  checked or billed again (its row keeps the first start URL that found it, even if that was another site's
  external link).
- Checks internal and (optionally) external links; external links are requested once and never crawled.
- Optional check of images, scripts, iframes and stylesheets (`includeResources`).
- Each row: `result` (`ok`, `redirect`, `broken`, or `blocked` for 401/403/429/999 answers, which usually mean a
  login wall, bot protection or rate limiting rather than a dead link), `httpStatus`, `redirectChain` (every hop with its status),
  `finalUrl`, `foundOn` (up to 10 pages), `foundOnCount`, `anchorText`, `linkType`, `error` (timeout, DNS lookup
  failed, redirect loop, too many redirects). `foundOn`/`foundOnCount` cover the pages seen linking to the URL by
  the time it was checked (the page that first linked to it is always included).
- Results stream into the dataset as links are checked: a timeout, abort or crash loses at most the link in flight.
- Every request has a hard time limit: a link that does not finish within twice `timeoutSecs` (all redirects and
  the page included) is reported as broken (timeout), so a server that trickles bytes cannot stall the run.
- Stops cleanly, without making requests it could not bill, when your maximum charge for the run is reached, about
  20 seconds before the run timeout (a request still in flight is cut off then, not reported and not billed), or
  when the run is aborted.
- Uses HEAD for links it does not need to read and re-checks with GET when a server answers HEAD with an error, so
  servers that mishandle HEAD are not reported as broken.
- A run summary (`SUMMARY` record) with counts per result, `redirectNotFollowed` (billed redirects whose destination
  was not checked), and `stopReason` (`charge_limit`, `timeout_soon`, `aborting`, `max_links`, or null when the crawl
  finished).

### Polite by design

- Identified user agent: `KelpForestLabs-LinkChecker/1.1 (+https://apify.com/kelp_forest_labs/broken-link-checker; ...)`.
- Honours robots.txt per host (RFC 9309: 4xx = no rules; 5xx, 429, timeouts and network errors = treat as
  disallow-all; a host that does not resolve or refuses connections is reported as broken) and `Crawl-delay` up to 10 s;
  redirects are only followed where the target host's robots.txt allows it (`redirectNotFollowed` says why);
  default delay 0.5 s between requests to the same host. Links robots.txt disallows are listed as free `skipped` rows.
- No logins, cookies, captchas or proxies. Private, loopback and cloud-metadata addresses are never requested.

### Pricing (pay per event)

| Event | Price |
|---|---|
| `link` (one unique link requested and reported) | US$0.0008 |

A link is billed once per run as a checked link, whatever the result: `ok`, `redirect`, `broken` or `blocked`
(finding broken links is the product), however many pages link to it. **`blocked` results (HTTP 401, 403, 429 or 999)
are billed as checked links:** the request was made and answered, even though the site refused it. **Not billed:** links skipped because of robots.txt, private/reserved targets, `mailto:`/`tel:`/`javascript:`
links, links beyond `maxLinks`, and an unusable start URL. A link that redirects into a path the target's robots.txt
disallows is billed as a checked link (`result: redirect`, `redirectNotFollowed` set): its own URL was requested and
answered, but the destination is not verified, so filter on `redirectNotFollowed` as well as `broken` when you look
for dead targets. Use `maxLinks` to cap spend (500 links = US$0.40).

Each run also has Apify's run-start charge of US$0.00005, once per run for up to 1 GB of memory plus once per extra GB (US$0.00005 at the default 512 MB),
charged when the run starts, even if every link is skipped.

### How to use

1. Open the actor in Apify Console and fill in the input form (or paste the JSON example below into the JSON editor).
2. Optional: set a maximum charge for the run under the run options to cap what a run can cost.
3. Click **Start**. Results appear in the **Output** tab as they are produced; export them as JSON, CSV, Excel or HTML.

You can also start runs from the Apify API or CLI, run the actor on a schedule (for example weekly), and send
results to other apps with Apify integrations and webhooks.

### Input example

```json
{"startUrls": [{"url": "https://example.com/"}], "maxDepth": 2, "maxPages": 50, "maxLinks": 500, "checkExternal": true}
```

### Output example (local fixture run, Oct 7, 2026)

```json
{"startUrl": "https://example.com/index.html", "url": "https://example.com/gone.html", "linkType": "internal", "tag": "a", "anchorText": "Gone",
 "foundOn": ["https://example.com/index.html", "https://example.com/about.html"], "foundOnCount": 2, "depth": 1,
 "status": "ok", "result": "broken", "httpStatus": 404, "finalUrl": "https://example.com/gone.html", "redirectChain": [], "redirectCount": 0, "error": null, "method": "GET"}
```

### FAQ

**Does it run JavaScript?**
No. Links added only by JavaScript are not seen.

**Is every link charged?**
Each unique link checked is charged once per run, whatever the result. Links skipped by robots.txt, private addresses, mailto:/tel: links and links beyond `maxLinks` are free.

**How do I limit cost?**
Use `maxLinks` (500 links = US$0.40) and `maxPages`, or set a maximum charge for the run: the checker stops before the first link it could not be paid for. Each run adds Apify's run-start charge (US$0.00005 once per run for up to 1 GB of memory, plus once per extra GB: US$0.00005 at the default 512 MB).

**What happens if the run times out or is aborted?**
Each link is saved the moment it is checked, so nothing checked is lost. The checker stops cleanly about 20 seconds before the run timeout, even in the middle of a slow request (that link is then not reported or billed), and writes its SUMMARY with the reason (`stopReason`). Each request also has a hard limit of twice `timeoutSecs`, so one slow server cannot stall the run.

**Why is a link `blocked` and not `broken`?**
HTTP 401, 403, 429 and 999 usually mean a login wall, bot protection or rate limiting: the page may work in a browser. These links were requested, so they are billed like any other result.

**Why is a page reported as ok when it shows 'not found'?**
Soft 404 pages that answer HTTP 200 look fine at the HTTP level; the checker does not judge page content.

### Limits: what this actor does not do

- It does not crawl other hosts, log in, run JavaScript or submit forms (links added only by JavaScript are not seen).
- It does not judge content (soft 404 pages that answer 200 are reported as `ok`).
- It is not a security scanner.
- Hosts that are down: a host that does not resolve or refuses connections is reported as `broken`. A host that
  resolves but never answers makes its robots.txt time out, and its links are then free `skipped` rows (not
  `broken`). If a host refused the robots.txt request and comes back later in the same run, its pages may be
  read without its robots.txt rules (rare; a known limitation).

### Changelog

- 1.1 (October 15): robots.txt is checked before every redirect hop; an unreachable or rate-limited (429)
  robots.txt now means "disallow everything" (free skipped rows), as RFC 9309 requires. `SUMMARY` now also counts
  `redirectNotFollowed`.
- 1.0 (October 2026): first public version. Update (October 13): a link to the bare domain (`https://site.com`)
  and to `https://site.com/` is now one link, checked and billed once (scheme/host case, default port and
  `#fragment` spellings are merged too; `/page` and `/page/` stay separate).
- Update (October 13, later): with several start URLs, a link reached from more than one of them is checked and
  billed once per run (its row keeps the first start URL that found it).
- Update (October 14): each start URL again crawls its own scope to `maxDepth` (pages already checked
  from another start URL are re-read for free so their links are found); a malformed start URL (for example a bad
  port) is a free error row.

# Actor input Schema

## `startUrls` (type: `array`):

Pages to start from. Only the start URL's own host is crawled; links to other hosts are checked but not crawled.

## `maxDepth` (type: `integer`):

1 = check the links on the start page only; 2 = also open the internal pages it links to and check their links; and so on.

## `maxPages` (type: `integer`):

Internal HTML pages opened to collect links.

## `maxLinks` (type: `integer`):

Total unique links requested in this run. Each one is billed as a checked link, whatever the result (ok, redirect, broken, or blocked for HTTP 401/403/429/999). Caps your spend.

## `checkExternal` (type: `boolean`):

Also check links that point to other hosts (one request each; never crawled).

## `includeResources` (type: `boolean`):

Also check img/script/iframe/source src and stylesheet/icon links.

## `delaySecs` (type: `number`):

Politeness delay. A larger robots.txt Crawl-delay (up to 10 s) takes precedence.

## `timeoutSecs` (type: `integer`):

A link that sends nothing for this long, or does not finish within twice this time (all redirects and the page included), is reported as broken (timeout).

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://docs.apify.com/"
    }
  ],
  "maxDepth": 1,
  "maxPages": 10,
  "maxLinks": 100,
  "checkExternal": true,
  "includeResources": false,
  "delaySecs": 0.5,
  "timeoutSecs": 15
}
```

# Actor output Schema

## `results` (type: `string`):

One row per checked or skipped link: result, HTTP status, redirect chain, final URL and the pages linking to it. Every checked link is billed, including blocked results (HTTP 401/403/429/999); skipped links are free.

## `summary` (type: `string`):

Counts per result (ok, redirect, broken, blocked, skipped) and why the run stopped (stopReason). ok, redirect, broken and blocked are billed checked links; skipped are free.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://docs.apify.com/"
        }
    ],
    "maxDepth": 1,
    "maxPages": 10,
    "maxLinks": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("kelp_forest_labs/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://docs.apify.com/" }],
    "maxDepth": 1,
    "maxPages": 10,
    "maxLinks": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("kelp_forest_labs/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://docs.apify.com/"
    }
  ],
  "maxDepth": 1,
  "maxPages": 10,
  "maxLinks": 100
}' |
apify call kelp_forest_labs/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kelp_forest_labs/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PAi9w6z7eHBgeYsmy/builds/uFOliW9PffeqYC9gT/openapi.json