# Broken Link Checker: Site Crawler with Source Pages (`creativefour/broken-link-crawler`) Actor

Crawl your whole website and find every broken link, redirecting link, and dead external link, with the exact page and link text where each one appears, so you know what to fix.

- **URL**: https://apify.com/creativefour/broken-link-crawler.md
- **Developed by:** [CreativeFour LLC](https://apify.com/creativefour) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$4.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does the Broken Link Checker do?

It **crawls your whole website** and finds every **broken link** (404, 410, 5xx, timeouts, dead domains), every **link that points at a redirect**, and every **dead external link**, and it tells you **exactly which page each one is on and what the link text says**. That's the part you need to actually fix it.

Enter your home page and click Start. Because it runs on Apify, you can also schedule it (for example, weekly), call it from the API, and connect the results to Slack, email, Google Sheets, or Zapier.

### Why use it?

- **Fix links, not just find them.** Every result lists the pages that contain the bad link, with the link text, so you can go straight to the page and edit it.
- **SEO hygiene.** Broken internal links waste crawl budget and leak link equity; links to redirects slow pages down and should point at the final URL.
- **Catch rot on other sites.** External links die quietly. Check them all in the same run.
- **Pay only for problems.** By default only broken, redirecting, and erroring links are saved and charged. Healthy links are counted in the summary for free.
- **Polite by default.** It follows robots.txt, limits concurrent requests to your server, and never crawls other websites.

### How to use it

1. Open the **Input** tab and enter your website's URL (for example `https://example.com`).
2. Optional: raise **Max pages to crawl** for a bigger site, or turn on **Check images, scripts, and stylesheets**.
3. Click **Start**.
4. Open the **Output** tab. The **Fix list** view gives one row per page to edit.
5. Export as CSV, Excel, JSON, or HTML, or schedule the run to repeat.

### Input

| Field | What it does |
|---|---|
| **Website to check** | Start URL(s). Same-site pages are followed; www and non-www count as one site. |
| **Max pages to crawl** | Crawl limit (default 500). Every link found on those pages is still checked. |
| **Report only problems** | On by default. Turn off to get a full link inventory. |
| **Check external links** | One lightweight request per external link. External sites are never crawled. |
| **Check images, scripts, and stylesheets** | Also check `img`, `script`, stylesheet, icon, video, audio, and iframe sources. |
| **Crawl subdomains** | Treat `blog.example.com` as part of `example.com`. |
| **Skip URLs matching** | Regular expressions or plain text, for example `/tag/` or `\?replytocom=`. |
| Advanced | Max depth, robots.txt, per-site and overall concurrency, timeout, max redirects, user agent. |

```json
{
  "startUrls": ["https://example.com"],
  "maxPages": 1000,
  "checkExternalLinks": true,
  "excludeUrlPatterns": ["/tag/", "\\?replytocom="]
}
```

### Output

One result per problem link. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

```json
{
  "url": "https://example.com/old-pricing",
  "isInternal": true,
  "finalStatus": 404,
  "statusClass": "4xx",
  "flags": ["broken"],
  "foundOnCount": 3,
  "foundOn": [
    { "page": "https://example.com/", "anchorText": "See pricing" },
    { "page": "https://example.com/blog/launch", "anchorText": "our plans" }
  ],
  "finalUrl": "https://example.com/old-pricing",
  "redirectCount": 0,
  "redirectChain": [],
  "responseTimeMs": 142,
  "error": null,
  "checkedAt": "2026-09-23T21:00:00.000Z"
}
```

Two views are ready in the Output tab:

- **Broken and redirecting links:** one row per bad link.
- **Fix list:** one row per page and link, which works as a to-do list for editing.

A **SUMMARY** record in the run's key-value store has the totals: pages crawled, link targets, checked, broken, redirected, errors, reported, and skipped by robots.txt.

### Data fields

| Field | Description |
|---|---|
| `url` | The link target |
| `foundOn` / `foundOnCount` | Pages that contain the link (up to 20) with link text, and the total number of occurrences |
| `isInternal` | Whether the link points to your own site |
| `finalStatus` / `statusClass` | HTTP status after redirects; `2xx` to `5xx`, or `error` |
| `finalUrl`, `redirectCount`, `redirectChain` | Where the link really goes, and every hop on the way |
| `flags` | `broken`, `redirect_chain`, `temporary_redirect`, `redirect_loop`, `too_many_redirects`, `https_to_http_downgrade`, `error` |
| `error` | Timeout, DNS failure, refused connection, or TLS error |

### Use it from AI agents (MCP)

AI agents can find and run this Actor through the [Apify MCP server](https://docs.apify.com/integrations/mcp).

- **Claude, ChatGPT, or any MCP client:** add `https://mcp.apify.com?tools=creativefour/broken-link-crawler` as a custom connector, and sign in to Apify when prompted.
- **Claude Code, Cursor, VS Code, or Codex:** run `apify mcp install claude-code` (swap in your client's name), then ask your agent to "check example.com for broken links with creativefour/broken-link-crawler".

The agent sends the same input as the Input tab, gets the same results, and pays the same per-result price.

### How much does it cost to check a website for broken links?

You pay per reported link: each problem link saved to the dataset is one result. Healthy links and crawled pages cost nothing extra. A typical small-business site with a few hundred pages and a handful of broken links costs cents. Set a **maximum charge per run** in the run options, and the crawler stops cleanly at that limit, keeping everything found so far.

### Tips

- Use **Skip URLs matching** for endless URL spaces such as calendars, tag archives, and faceted filters.
- Keep **Max concurrent requests per site** low (the default is 3) on small or shared hosting.
- Links that only **redirect** still work, but updating them to the final URL makes pages faster and cleaner for SEO.
- Schedule a weekly run and send new problems to Slack or email with an Apify integration.

### FAQ and support

**Does it render JavaScript?** No. It reads the HTML your server sends, which covers most websites, and it doesn't find links that are only added by client-side JavaScript.

**Will it crawl other websites?** No. Only your site's pages are crawled. External links get a single status check each.

**Is it okay to crawl any site?** Please crawl sites you own or are allowed to test. The crawler follows robots.txt and limits its request rate by default.

**Found a bug or need a feature?** Open an issue on the **Issues** tab. Custom versions are available on request.

# Actor input Schema

## `startUrls` (type: `array`):

The home page (or any page) of the site to crawl. Every page on the same site is followed; www and non-www count as the same site.

## `maxPages` (type: `integer`):

Stop crawling after this many pages. Links found on those pages are still all checked.

## `onlyProblems` (type: `boolean`):

Save only broken, redirecting, or erroring links (you pay only for these). Turn off to get every link.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites (one lightweight request each; external sites are never crawled).

## `includeResources` (type: `boolean`):

Also check img, script, stylesheet, icon, video, audio, and iframe sources.

## `includeSubdomains` (type: `boolean`):

Treat subdomains (blog.example.com) as part of the site.

## `excludeUrlPatterns` (type: `array`):

Regular expressions (or plain text) for URLs to ignore, for example /tag/ or ?replytocom=

## `maxDepth` (type: `integer`):

How many clicks away from the start page to crawl.

## `respectRobotsTxt` (type: `boolean`):

Don't crawl or check pages of your site that robots.txt disallows.

## `perHostConcurrency` (type: `integer`):

Keeps the crawl polite to your server.

## `maxConcurrency` (type: `integer`):

Across all sites when checking external links.

## `timeoutSecs` (type: `integer`):

How long to wait for each request before recording a timeout error.

## `maxRedirects` (type: `integer`):

Stop following a redirect chain after this many hops.

## `userAgent` (type: `string`):

The User-Agent header sent with every request, and the name robots.txt rules are matched against.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawler-test.com/"
  ],
  "maxPages": 10,
  "onlyProblems": true,
  "checkExternalLinks": true,
  "includeResources": false,
  "includeSubdomains": false,
  "maxDepth": 10,
  "respectRobotsTxt": true,
  "perHostConcurrency": 3,
  "maxConcurrency": 10,
  "timeoutSecs": 15,
  "maxRedirects": 10,
  "userAgent": "Mozilla/5.0 (compatible; BrokenLinkCrawler/1.0; +https://apify.com)"
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/"
    ],
    "maxPages": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("creativefour/broken-link-crawler").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://crawler-test.com/"],
    "maxPages": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("creativefour/broken-link-crawler").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/"
  ],
  "maxPages": 10
}' |
apify call creativefour/broken-link-crawler --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,creativefour/broken-link-crawler"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ryIcBtFqxnbGP8Fwd/builds/1b7tndNy5ezpEBxVF/openapi.json