# Broken Link Checker & Redirect Chain Auditor (`grace_hartwell/broken-link-redirect-auditor`) Actor

Crawl a website to find broken links (404, 5xx), redirect chains, redirect loops, timeouts and mixed content. Polite, robots.txt-aware link audit.

- **URL**: https://apify.com/grace_hartwell/broken-link-redirect-auditor.md
- **Developed by:** [Grace M. Hartwell](https://apify.com/grace_hartwell) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 link checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

<img src="https://apify-image-uploads-prod.s3.us-east-1.amazonaws.com/dpsKHwXKHdbwBgwOT-profile-dxtJoi0ame-avatar-hartwell-800.png" alt="Hartwell Toolworks" width="96" height="96" />

**Broken Link Checker & Redirect Chain Auditor** crawls your website and **finds broken links (404, 410, other 4xx, 5xx), redirect chains, redirect loops, timeouts and insecure http / mixed-content links**. It checks every internal and external link **once** (HEAD first, GET as a fallback) and records every redirect hop. You get one clear report row per link, showing **the pages where the link appears**, so you know exactly what to fix. The crawler respects robots.txt and per-host limits, and it costs **$0.50 per 1,000 links checked**.

### What does the broken link checker do?

1. **Crawls** the internal HTML pages of your site from a start URL, within a maximum depth and page count. robots.txt is respected and crawling is rate-limited.
2. **Collects** every link (`<a>`, `<area>`) and, optionally, every image, script, stylesheet, iframe and audio/video source, together with the pages and anchor texts where it appears.
3. **Checks** each unique URL once: `HEAD` first, then `GET` when a server answers HEAD with an error code. Redirects are followed **manually**, so every hop is recorded.
4. **Retries** 429, 5xx and network errors with exponential backoff. Timeouts are not retried, which keeps runs fast.
5. **Reports** one dataset item per link (internal links first, then external) and saves a run summary to the `OUTPUT` record.

### What does it find?

| Finding | `status` / issue code |
| --- | --- |
| 404, 410 and other 4xx, 5xx responses | `broken` / `BROKEN` (with the final HTTP status) |
| Redirects, and **redirect chains** of 2+ hops (every hop listed) | `redirect` / `REDIRECT`, `REDIRECT_CHAIN` |
| **Redirect loops** (A → B → A) | `redirect-loop` / `REDIRECT_LOOP` |
| More redirects than `maxRedirects` | `too-many-redirects` / `TOO_MANY_REDIRECTS` |
| No answer within the timeout | `timeout` / `TIMEOUT` |
| DNS, TLS and connection errors, invalid `Location` headers | `error` / `ERROR` (e.g. `ENOTFOUND` in `error`) |
| https page linking to an `http://` URL | `INSECURE_HTTP_LINK` |
| https page loading an `http://` image, script, CSS, iframe or media file | `MIXED_CONTENT` (needs `checkResources`) |
| Redirect from https to http | `HTTPS_TO_HTTP_REDIRECT` |
| http link that redirects to https | `HTTP_UPGRADED_BY_REDIRECT` (update the link) |
| Internal URL disallowed by robots.txt | `skipped-robots` / `ROBOTS_TXT_DISALLOWED` (not requested, not charged) |

### Key features

- 🔗 **Internal and external link checking**: external sites get one HEAD/GET check per unique URL and are never crawled.
- ↪️ **Full redirect chains**: each hop with its URL, status code and `Location`, plus the final URL and status.
- 📍 **Where each link appears**: up to 10 pages with anchor text and tag (`foundOn`), plus the total number of occurrences (`foundOnCount`).
- 🔒 **HTTPS hygiene**: finds insecure `http://` links, mixed content, and https → http redirects.
- 🤝 **Polite crawling**: robots.txt is respected for your site, at most 2 parallel requests per host by default, rate limits for crawling, and a clear user agent (`BrokenLinkRedirectAuditor/1.0`).
- 🧾 **Issues-only mode**: `onlyIssues` keeps healthy links out of the dataset (they are still checked and counted).
- 💸 **Budget control**: `maxLinksToCheck` and *Max total charge* stop the run cleanly.

### Use cases

1. **Technical SEO audits**: find 404s and long redirect chains that waste crawl budget and link equity.
2. **Website migrations**: check that old URLs redirect in **one hop** to the right target, with no loops or https → http downgrades.
3. **Content QA before a launch**: catch dead internal links, missing images and broken scripts before visitors do.
4. **Scheduled link monitoring**: run weekly on a blog, help center or documentation site with an Apify schedule.
5. **HTTPS clean-up**: list remaining `http://` links and mixed-content resources on https pages.
6. **Outbound link hygiene**: find external links that now return 404 or redirect somewhere else.

### How much does it cost to check broken links?

The Actor uses **pay-per-event** pricing:

| Event | Price | When it is charged |
| --- | --- | --- |
| `link-checked` | **$0.0005** per unique URL checked ($0.50 per 1,000) | After a link has been checked (crawling pages is included) |
| `apify-actor-start` | $0.00005 per run | Automatically at run start (Apify default price) |

Each URL is checked and charged **once per run**, no matter how many pages link to it. URLs skipped because of robots.txt are free. Healthy links hidden by `onlyIssues` are still checked, so they are still charged.

**Example calculation: a 200-page site with 1,500 unique links:**

```
1,500 unique links × $0.0005   = $0.75
+ 1 Actor start × $0.00005     = $0.00005   (default 512 MB memory → 1 start event)
-------------------------------------------------
Total                          ≈ $0.75005   (= $0.50 per 1,000 links)
```

### How to find broken links on your website

1. Open **Broken Link Checker & Redirect Chain Auditor** in Apify Store and click **Try for free**.
2. Enter your **Website (start URL)**, e.g. `https://www.example.org/`.
3. Set **Max pages to crawl** and **Max crawl depth** (0 = only the start page).
4. Choose what to check: **external links**, **images, scripts, stylesheets and iframes** (needed for mixed content), and whether to **only output links with issues**.
5. Optional: set **Max links to check** or a *Max total charge* to limit the cost.
6. Click **Start**. Open the **Overview** table, or export the report as CSV, Excel or JSON. The summary is in the `OUTPUT` record.

### Input example

```json
{
    "startUrl": "https://www.example.org/",
    "maxPages": 200,
    "maxDepth": 3,
    "includeSubdomains": false,
    "checkExternalLinks": true,
    "checkResources": true,
    "onlyIssues": true,
    "maxLinksToCheck": 0,
    "requestTimeoutSecs": 15,
    "maxRedirects": 10,
    "maxRetries": 2,
    "maxConcurrency": 5,
    "perHostConcurrency": 2,
    "maxRequestsPerMinute": 120
}
```

| Input field | Type | Default | Description |
| --- | --- | --- | --- |
| `startUrl` | string | `https://example.com/` | Page to start from. Only pages on this host are crawled. Replace the default with your own website. |
| `maxPages` | integer | `50` | Internal HTML pages to crawl (1–10,000). |
| `maxDepth` | integer | `3` | 0 = only the start page, 1 = plus the pages it links to, … (0–20). |
| `includeSubdomains` | boolean | `false` | Treat `blog.example.com` as internal when starting at `example.com`. |
| `checkExternalLinks` | boolean | `true` | Also check links to other websites (never crawled). |
| `checkResources` | boolean | `false` | Also check images, scripts, stylesheets, iframes and media; detects mixed content. |
| `onlyIssues` | boolean | `false` | Leave healthy links out of the dataset. |
| `maxLinksToCheck` | integer | `0` | 0 = no limit; also capped by *Max total charge*. |
| `requestTimeoutSecs` | integer | `15` | Timeout per request (3–120). |
| `maxRedirects` | integer | `10` | Longer chains become `too-many-redirects` (1–30). |
| `maxRetries` | integer | `2` | Retries for 429, 5xx and network errors (0–6). |
| `maxConcurrency` | integer | `5` | Parallel crawl requests (1–10); link checks use up to 2× this. |
| `perHostConcurrency` | integer | `2` | Max parallel link checks per host (1–5). |
| `maxRequestsPerMinute` | integer | `120` | Rate limit for crawling your site (1–1,000). |

### Output example

One dataset item per unique link (example based on a local test run; the host has been replaced):

```json
{
    "url": "https://www.example.org/chain-1",
    "status": "redirect",
    "httpStatus": 301,
    "finalUrl": "https://www.example.org/ok",
    "finalStatus": 200,
    "redirectCount": 3,
    "redirectChain": [
        { "url": "https://www.example.org/chain-1", "status": 301, "location": "https://www.example.org/chain-2" },
        { "url": "https://www.example.org/chain-2", "status": 302, "location": "https://www.example.org/chain-3" },
        { "url": "https://www.example.org/chain-3", "status": 307, "location": "https://www.example.org/ok" }
    ],
    "internal": true,
    "kind": "anchor",
    "issues": ["REDIRECT", "REDIRECT_CHAIN"],
    "mixedContentOn": [],
    "insecureLinkOn": [],
    "foundOn": [{ "page": "https://www.example.org/", "text": "Redirect chain", "tag": "a" }],
    "foundOnCount": 1,
    "method": "HEAD",
    "responseTimeMs": 12,
    "error": null,
    "checkedAt": "2026-10-25T22:59:05.500Z"
}
```

Robots.txt-skipped item (free):

```json
{ "url": "https://www.example.org/private/secret", "status": "skipped-robots", "internal": true, "issues": ["ROBOTS_TXT_DISALLOWED"], "foundOn": [{ "page": "https://www.example.org/", "text": "Private", "tag": "a" }], "foundOnCount": 1, "note": "Not requested: disallowed by robots.txt. Not charged." }
```

**Run summary** (`OUTPUT` record, numbers from the local test site):

```json
{
    "startUrl": "https://www.example.org/",
    "pagesCrawled": 8,
    "uniqueLinksFound": 17,
    "linksChecked": 16,
    "robotsSkipped": 1,
    "byStatus": { "skipped-robots": 1, "ok": 8, "broken": 4, "redirect": 1, "redirect-loop": 1, "timeout": 2 },
    "byIssue": { "BROKEN": 4, "REDIRECT": 1, "REDIRECT_CHAIN": 2, "REDIRECT_LOOP": 1, "TIMEOUT": 2 },
    "stoppedEarly": false
}
```

#### Output fields

| Field | Description |
| --- | --- |
| `url` | The checked URL (fragment removed). |
| `status` | `ok`, `redirect`, `broken`, `timeout`, `error`, `redirect-loop`, `too-many-redirects`, `skipped-robots`. |
| `httpStatus` | Status of the first response (e.g. 301), or `null`. |
| `finalUrl`, `finalStatus` | Where the redirects end and that page's status. |
| `redirectCount`, `redirectChain` | Number of hops and `{ url, status, location }` per hop. |
| `internal` | `true` if the URL is on your site (subdomains only with `includeSubdomains`). |
| `kind` | `anchor` (link) or `resource` (image, script, CSS, iframe, media). |
| `issues` | Issue codes, e.g. `BROKEN`, `REDIRECT_CHAIN`, `MIXED_CONTENT`. |
| `mixedContentOn`, `insecureLinkOn` | Up to 10 https pages that load the resource over http / link to it over http. |
| `foundOn`, `foundOnCount` | Up to 10 pages with anchor text and tag; total number of occurrences. |
| `method` | `HEAD` or `GET` (the method used for the last request). |
| `responseTimeMs` | Total time for the check, including redirects and retries. |
| `error` | Readable error, e.g. `HTTP 404` or `HTTP 403 (access denied – may be login/bot protection)`. |
| `checkedAt` | Timestamp. |

### Integrations and API

The report is a standard Apify dataset, so you can use the Apify API, API clients, schedules, webhooks and integrations (for example Make, Zapier, n8n or Google Sheets exports).

```bash
curl -X POST "https://api.apify.com/v2/acts/grace_hartwell~broken-link-redirect-auditor/run-sync-get-dataset-items?token=[YOUR_APIFY_TOKEN]" \
  -H "Content-Type: application/json" \
  -d '{"startUrl":"https://www.example.org/","maxPages":50,"onlyIssues":true}'
```

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('grace_hartwell/broken-link-redirect-auditor').call({ startUrl: 'https://www.example.org/', onlyIssues: true });
const { items } = await client.dataset(run.defaultDatasetId).listItems();
const broken = items.filter((i) => i.status === 'broken');
```

### FAQ

#### Why is a link that works in my browser reported as 403?

Some sites block non-browser clients (bot protection) or require a login. The item then says `HTTP 403 (access denied – may be login/bot protection)`. Check these links manually.

#### Does the link checker render JavaScript?

No. Pages are fetched as HTML without a browser, so links that are injected only by client-side JavaScript are not found.

#### How is a redirect chain counted?

`redirectCount` is the number of redirect hops. One hop is reported as `REDIRECT`; two or more hops also raise `REDIRECT_CHAIN`.

#### Are external websites crawled?

No. External links get a single check (HEAD, with a GET fallback when needed) per unique URL. Only pages on your start host (and, optionally, its subdomains) are crawled.

#### Does it check image, script and CSS links?

Yes, if you turn on `checkResources`. This is also needed to detect mixed content.

#### How do I keep the cost under control?

Use `maxPages`, `maxDepth`, `maxLinksToCheck` or a *Max total charge*. The Actor checks internal links first and stops cleanly at the limit; `stoppedEarly` in the summary tells you whether the limit was hit.

#### Is `www.example.com` treated as the same site as `example.com`?

Yes. A leading `www.` is ignored when deciding whether a URL is internal.

### Limitations: what this Actor does not do

- **No JavaScript rendering**: links added only by client-side scripts are missed.
- **No logins, form submissions or bot-protection bypass.** Protected pages may show up as 401/403 `broken`.
- **Fragments are not checked**: `#anchor` parts are removed, so missing in-page anchors are not detected.
- **Only http(s) links**: `mailto:`, `tel:`, `javascript:`, `data:`, `ftp:` and similar schemes are ignored.
- **No sitemap import**: pages are discovered only by following links from the start URL, within `maxDepth` and `maxPages`.
- **Files are checked but not crawled**: URLs ending in `.pdf`, `.zip`, images, `.css`, `.js` and similar are checked, but not parsed for more links.
- **robots.txt applies to your site only**: external links are checked without reading their robots.txt (one request per unique URL). robots.txt rules are read from the start host only.
- **Timeouts can take up to about twice `requestTimeoutSecs`**: when the HEAD request times out, a GET request is tried as well, and timeouts are not retried.
- **Link checks are not limited per minute**: `maxRequestsPerMinute` applies to crawling, while link checks are limited by `perHostConcurrency` and the number of workers.
- **Crawled pages are requested twice** (once to crawl and once to check), but they are charged once.
- **No content extraction**: apart from links, anchor text and tags, nothing is extracted from your pages.

***

Author: Grace M. Hartwell

# Actor input Schema

## `startUrl` (type: `string`):

The page to start from, e.g. https://example.com/. Only pages on this host are crawled.

## `maxPages` (type: `integer`):

Internal HTML pages to crawl for links.

## `maxDepth` (type: `integer`):

0 = only the start page, 1 = start page + pages it links to, …

## `includeSubdomains` (type: `boolean`):

Crawl blog.example.com when starting at example.com.

## `checkExternalLinks` (type: `boolean`):

Also check links to other websites (one HEAD/GET request each, never crawled).

## `checkResources` (type: `boolean`):

Also check sub-resources and detect mixed content (http resources on https pages).

## `onlyIssues` (type: `boolean`):

Skip healthy links in the dataset (they are still checked and counted).

## `maxLinksToCheck` (type: `integer`):

0 = no limit. Also capped by your 'Max total charge'.

## `requestTimeoutSecs` (type: `integer`):

A link that does not answer within this time is reported as timeout.

## `maxRedirects` (type: `integer`):

Longer chains are reported as too-many-redirects.

## `maxRetries` (type: `integer`):

With exponential backoff. Timeouts are not retried.

## `maxConcurrency` (type: `integer`):

Parallel crawl requests (link checks use up to 2×, but never more than 'per host' on one host).

## `perHostConcurrency` (type: `integer`):

Keeps the load on each website low.

## `maxRequestsPerMinute` (type: `integer`):

Rate limit for crawling your site.

## Actor input object example

```json
{
  "startUrl": "https://example.com/",
  "maxPages": 50,
  "maxDepth": 3,
  "includeSubdomains": false,
  "checkExternalLinks": true,
  "checkResources": false,
  "onlyIssues": false,
  "maxLinksToCheck": 0,
  "requestTimeoutSecs": 15,
  "maxRedirects": 10,
  "maxRetries": 2,
  "maxConcurrency": 5,
  "perHostConcurrency": 2,
  "maxRequestsPerMinute": 120
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://example.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("grace_hartwell/broken-link-redirect-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrl": "https://example.com/" }

# Run the Actor and wait for it to finish
run = client.actor("grace_hartwell/broken-link-redirect-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://example.com/"
}' |
apify call grace_hartwell/broken-link-redirect-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,grace_hartwell/broken-link-redirect-auditor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/li26EkxITqRHwAeOd/builds/1rSkmUayM6Im5ba46/openapi.json