# Broken Link Checker: 404s, Redirects & Anchor Text (`blond_terrarium/broken-link-checker-pro`) Actor

Crawl a website and find broken links (404, 5xx, dead domains), redirect chains and blocked links, with the source page and anchor text for every link. Polite, respects robots.txt. Pay only per link that got a definitive result.

- **URL**: https://apify.com/blond_terrarium/broken-link-checker-pro.md
- **Developed by:** [Mogul Tools](https://apify.com/blond_terrarium) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 link checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker: Find 404s, Dead Domains & Redirect Chains with Source Page + Anchor Text

Crawl your website and get a clean report of **every broken link**, showing **which page it's on, what the anchor text says**, and the full **redirect chain**. Internal and external links are both checked. It's polite by default and respects robots.txt. **You pay only for links that got a definitive answer.** Timeouts, connection errors and robots-skipped links are listed but free.

Use it as a **website broken link checker**, a **dead link finder / 404 checker**, a **redirect chain checker** for SEO audits, or an automated **link-rot monitor** you schedule monthly. Export to CSV/Excel and hand the "fix list" straight to whoever edits the site.

### Quick start (about 1 minute)

1. Enter your homepage in **Website URL**.
2. Set **Max pages to crawl** (100 is a good start) and choose **Rows to output** → *Problems only*.
3. Click **Start**. Export the table: each row tells you the broken URL, the page it's on, and the anchor text to search for.

### What can it do?

- 🕷️ **Same-domain crawl** from a start URL, up to *Max pages* (optionally including subdomains)
- 🔗 **Checks every link** it finds: internal pages, external sites (HEAD with an automatic GET fallback), and optionally images, scripts and stylesheets
- ❌ **Broken detection**: 4xx/5xx, dead domains (DNS), TLS errors, redirect loops and overly long redirect chains
- ↪️ **Redirect chains** hop by hop (`301 → 302 → 200`), so you can fix links that point to redirects
- 📍 **Where to fix it**: source page and anchor text for each link, up to 20 occurrences, plus a full count
- 🚧 **Honest "blocked" status**: sites that refuse bots (401/403/429/999, LinkedIn for example) are labeled *blocked*, not falsely *broken*
- 🤝 **Polite**: per-host delay (500 ms default), honors robots.txt `Crawl-delay`, `Retry-After` and `nofollow`, and uses an honest User-Agent
- 📊 **SUMMARY** record with totals and a broken-links list for quick reporting

### Who uses it?

- **Site owners & SEOs**: fix 404s that hurt rankings and user trust
- **Agencies**: monthly link audits for client sites (schedule it in Apify)
- **Content teams & docs maintainers**: catch outdated external references
- **Developers/CI**: run after each deploy through the API and fail the build if the SUMMARY shows `broken > 0`

### Popular recipes

| Goal | Input |
|---|---|
| Quick 404 audit of a small site | `"maxPages": 100, "reportMode": "problems"` |
| Only the links you must fix | `"reportMode": "broken"` |
| Internal links only (fast) | `"checkExternalLinks": false` |
| Broken images too | `"checkImages": true` |
| Skip a section (e.g. /tag/ pages) | `"excludeUrlPatterns": ["/tag/"]` |
| Monthly link-health report per client | Save a Task per site + monthly Schedule |

### How much does it cost?

Pay per unique link checked: **$0.80 per 1,000** on Free, $0.70 on Starter, $0.60 on Scale, and $0.50 on Business. There are no platform-usage fees on top. A 100-page site typically has 300 to 1,500 unique links, which comes to about **$0.25 to $1.20** per full audit.

Every checked link is billed once, even when **Rows to output** hides it (for example "Broken only"). Links that timed out, errored, or were skipped by robots.txt are never billed. Set **Max links** or **Max total charge** to cap spend.

### Input

```json
{
  "startUrl": "https://crawler-test.com/",
  "maxPages": 100,
  "checkExternalLinks": true,
  "checkImages": false,
  "reportMode": "problems"
}
```

### Output (one row per unique link)

```json
{
  "url": "https://crawler-test.com/redirects/invalid_redirect",
  "result": "broken",
  "broken": true,
  "statusCode": 303,
  "finalStatusCode": 404,
  "finalUrl": "https://crawler-test.com/images/foo.jpg",
  "redirected": true,
  "redirectChain": [
    {"url": "https://crawler-test.com/redirects/invalid_redirect", "status": 303},
    {"url": "https://crawler-test.com/images/foo.jpg", "status": 404}
  ],
  "isInternal": true,
  "linkType": "a",
  "foundOn": "https://crawler-test.com/",
  "anchorText": "Redirect 303 To 404",
  "sourceCount": 1,
  "error": "HTTP 404",
  "charged": true
}
```

`result` is one of `ok`, `redirect`, `broken`, `blocked`, `error` (inconclusive, free) or `skipped` (robots.txt, free). Rows are sorted with broken links first.

### Use it from code or CI (API)

```bash
curl -X POST "https://api.apify.com/v2/acts/blond_terrarium~broken-link-checker-pro/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrl":"https://example.com","maxPages":50,"reportMode":"broken"}'
```

An empty array means no broken links, so a CI step can simply fail when the response has any rows.

Python:

```python
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("blond_terrarium/broken-link-checker-pro").call(run_input={"startUrl": "https://example.com", "reportMode": "broken"})
summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("SUMMARY")["value"]
print(summary)
```

### Use it from AI agents (MCP)

Available through the Apify MCP server (`https://mcp.apify.com`). Ask Claude, Cursor or ChatGPT: *"Check example.com for broken links with blond_terrarium/broken-link-checker-pro and list the pages I need to fix."* The output (URL, source page, anchor text, status) is easy for an agent to turn into a fix list or a ticket.

### Integrations

Schedules, webhooks, Slack, email, Google Sheets, Zapier, Make and n8n all work through Apify. Popular setup: monthly Schedule → Slack/email alert with the SUMMARY.

### How it compares

| | This Actor | Desktop crawlers | Free online checkers |
|---|---|---|---|
| Runs in the cloud, schedulable, API | ✅ | ❌ your laptop | limited |
| Source page **and anchor text** per broken link | ✅ | ✅ | often ❌ |
| Full redirect chain hop by hop | ✅ | ✅ | rarely |
| Bot-blocked links labeled *blocked*, not *broken* | ✅ | ❌ (false positives) | ❌ |
| Pay per link checked, inconclusive results free | ✅ | licence fee | page caps / ads |

### FAQ

**Why "blocked" and not "broken"?** Many big sites reject automated requests (403/429/999) even though the link works fine in a browser. We don't want you "fixing" links that aren't broken.

**How long does a run take?** A 100-page site usually finishes in a few minutes. Large sites scale with *Max pages* and the polite per-host delay.

**What does a full audit cost?** See the pricing section above; a 100-page site is typically about $0.25 to $1.20.

**Can it check pages behind a login?** No. It's for public pages only.

**Will it hammer my server?** No. Requests to one host are spaced out (500 ms default, or your robots.txt Crawl-delay), and 429/503 responses trigger a backoff.

**Can I check a list of URLs instead of crawling?** Not yet: it crawls from one start URL. To get a site's full URL list, use [Sitemap URL Extractor](https://apify.com/blond_terrarium/sitemap-url-extractor-pro). For page visuals, see [Bulk Website Screenshot](https://apify.com/blond_terrarium/bulk-website-screenshot).

# Actor input Schema

## `startUrl` (type: `string`):

Page to start crawling from. Only pages on this domain are crawled; links to other sites are checked but not crawled.

## `maxPages` (type: `integer`):

Internal HTML pages to crawl for links. Links found beyond this are still checked.

## `maxLinks` (type: `integer`):

Stop queuing new links after this many unique links. Also use 'Max total charge' to cap spend.

## `checkExternalLinks` (type: `boolean`):

Check links to other websites (HEAD request, GET fallback). They are never crawled.

## `includeSubdomains` (type: `boolean`):

Crawl blog.example.com etc. as part of example.com.

## `checkImages` (type: `boolean`):

Also check image URLs (img src).

## `checkScriptsAndStyles` (type: `boolean`):

Also check JavaScript (script src) and CSS stylesheet URLs.

## `reportMode` (type: `string`):

Which rows go to the dataset. Every checked link is still billed once; the SUMMARY record always has full totals.

## `maxSourcesPerLink` (type: `integer`):

How many (page, anchor text) occurrences to list per link. sourceCount always has the full count.

## `excludeUrlPatterns` (type: `array`):

Links matching any pattern are neither crawled nor checked (e.g. /logout, ?sort=).

## `respectRobotsTxt` (type: `boolean`):

Skip URLs disallowed by robots.txt (reported, not charged) and honor Crawl-delay. Turn off only for sites you own.

## `requestDelayMs` (type: `integer`):

Politeness delay per host. A larger robots.txt Crawl-delay wins (max 10 s); 429/503 Retry-After is honored.

## `maxConcurrency` (type: `integer`):

Across all hosts. Requests to one host are still spaced by the delay.

## `requestTimeoutSecs` (type: `integer`):

Timeouts are reported as 'error' and not charged.

## `maxRetries` (type: `integer`):

Retries for timeouts, connection resets, 429 and 5xx.

## `maxRedirects` (type: `integer`):

Longer chains are reported as broken.

## `userAgent` (type: `string`):

Defaults to an honest bot User-Agent (BrokenLinkChecker/1.0).

## `proxyConfiguration` (type: `object`):

Optional. Only needed if a site blocks datacenter IPs. Proxy traffic is billed by Apify separately.

## Actor input object example

```json
{
  "startUrl": "https://crawler-test.com/",
  "maxPages": 100,
  "maxLinks": 0,
  "checkExternalLinks": true,
  "includeSubdomains": false,
  "checkImages": false,
  "checkScriptsAndStyles": false,
  "reportMode": "all",
  "maxSourcesPerLink": 20,
  "excludeUrlPatterns": [],
  "respectRobotsTxt": true,
  "requestDelayMs": 500,
  "maxConcurrency": 5,
  "requestTimeoutSecs": 20,
  "maxRetries": 1,
  "maxRedirects": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

One dataset row per checked link (status, source page, error).

## `summary` (type: `string`):

SUMMARY record with run totals, stored in the default key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://crawler-test.com/",
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("blond_terrarium/broken-link-checker-pro").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrl": "https://crawler-test.com/",
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("blond_terrarium/broken-link-checker-pro").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://crawler-test.com/",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call blond_terrarium/broken-link-checker-pro --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,blond_terrarium/broken-link-checker-pro"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/otd1xbe6JSQiwoE8x/builds/FHoO960JwK1A8JBhW/openapi.json