# Broken Link Checker – Find 404s & Redirect Chains (`luxe_tasks/broken-link-checker`) Actor

Crawl a site you own and find broken internal and external links (4xx, 5xx, DNS errors, timeouts), redirect chains and the pages each link is on. Price: $0.001 per link checked ($1 per 1,000) + $0.0005 per crawled page of your site (each page once, not also as a link) + $0.005 start fee per run.

- **URL**: https://apify.com/luxe_tasks/broken-link-checker.md
- **Developed by:** [Luxe Tasks](https://apify.com/luxe_tasks) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 link checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link & Redirect Checker

Developer: Luxe_tasks.

Give it one or more pages of **a site you own or are authorized to audit**. It crawls that site (same registered domain only, within your page
and depth limits), checks **every link it finds**, internal and external, and reports for each link: the **HTTP status**, the **redirect chain**
(every hop with its status and `Location` resolved to a full URL (when valid), and the final URL), whether it is **broken** (4xx, 5xx, redirect loop, DNS failure, timeout), the
**pages where it was found** (up to 3) and its **anchor text**. Links it could not verify (bot walls, rate limits, robots.txt) are reported as
such, never as broken, and have no per-link or per-page charge (the per-run start fee still applies).

### How it checks

- **Your site** (same registered domain as the start URL, for example `www.example.org` and `blog.example.org` for `example.org`): pages are
  opened with GET and their `<a href>` links collected, breadth-first, up to **Max link depth** clicks from the start URL. Same-site links beyond
  the depth limit get one lightweight check (below), with redirects followed inside the site.
- **Other sites** (external links): **one lightweight request** per link: `HEAD`, or a 1-byte ranged `GET` if the server does not support HEAD
  (405/501). A redirect is reported (status, `Location`) and **not followed**. External sites are never crawled.
- Each unique link is checked once per start URL (fragments `#...` removed), however many pages link to it. `mailto:`, `tel:`, `javascript:` and
  similar links are ignored.
- Compressed pages (gzip, deflate, brotli) are decoded. Pages are read up to **5 MB**; links after that point are not seen (the report flags the page with `pageTruncated`).
- No JavaScript is executed: links that only exist after client-side rendering are not seen. Sitemaps are not read in this version.

### Input

- `startUrls` (required, up to 100): one report per start URL. URLs with a user name or password (`user:pass@host`) are refused; query strings are never logged.
- `maxPages` (default 500, 1-5,000): most same-site URLs requested **in the whole run** (pages crawled plus same-site links checked without crawling; the redirects of one URL count as that URL). Links left over are listed as `not_checked_page_cap` and have no per-link charge.
- `maxDepth` (default 5, 0-10): clicks from the start URL to crawl. `0` opens only the start URLs and checks their links.
- `checkExternalLinks` (default on): check links to other sites (one lightweight request each, plus one robots.txt request per external site). Off: they are listed as `not_checked_external_off`, not requested and have no per-link charge.
- `timeoutSecs` (default 15, 2-60): how long to wait for one response; a link that does not answer in time is reported as `timeout` (broken, no per-link charge).
- `maxConcurrency` (default 3, at most 3): start URLs processed in parallel. Each site still gets at most about 1 request per second.

Each run has a default timeout of 1 hour (3600 seconds); for very large lists you can raise the timeout in the run options.
If the run is about to reach its timeout, the Actor stops starting new requests, saves what it has, and lists the rest as `not_checked_time_limit` (no per-link charge).

### Output

- **Dataset**: one record per start URL (query strings shown as `?[redacted]`): `index, url, status, ok, charged, chargedEvents, chargedByEvent, httpStatus, error, robots, timingMs` plus
  `startVerdict`, `pagesCrawled`, `uniqueLinks`, `linksChecked` (charged `link-checked` events), `pagesCharged` (charged `page-crawled` events), `brokenCount`, `redirectedCount`, `notCheckedCount`, `internalLinks`, `externalLinks`,
  `pageCapReached`, `budgetReached`, `timeLimitReached`, `stoppedSites`, `verdicts` (count per verdict), and `brokenLinks` (first 100: `url, status, verdict, foundOn, anchorText`). The **Broken links** view lists them one per row.
- **Full report** (key-value store, `link-report-<index>-<host>-<hash>-json`, linked from the record): every unique link with `url, internal, crawled, verdict, broken, billed, billedEvent, status, method,
  redirectChain [{url, status, location}], finalUrl, redirectNotFollowed, foundOn, occurrences, anchorText, error`. The report keeps full URLs, including query strings.
  Above 8 MB, entries of links that are simply OK (200, no redirect) are left out and counted in `okLinksOmitted`.
- `RUN_SUMMARY` (key-value store): counts, charged events (a start URL charged both events is listed as `page-crawled*N+link-checked*M`), stopped domains.
  To compare with Apify's billing view, `chargedByEvent` shows the charges this Actor recorded per event, for example `{"page-crawled": 10, "link-checked": 9}`: in each dataset record (that start URL), in RUN_SUMMARY's `itemCharges` (one entry per record saved since the last start or resume) and as RUN_SUMMARY's total (charges made since the last start or resume, so after a restart it can be lower than what Apify billed for the whole run; `chargeUncertainByEvent` holds events whose charge result is unknown). `STOPPED_SITES`: sites stopped during the run. Apify's billing is the official record.

#### Verdicts

| Verdict | Meaning | Broken | Charge (link or crawled page) |
|---|---|---|---|
| `ok` | 2xx (after any redirects inside your site; see `redirectChain`) | no | yes |
| `redirect` | external link answered with a redirect (not followed; `finalUrl` = its target) | no | yes |
| `redirect_offsite` | same-site link redirects to another site (not followed) | no | yes |
| `auth_required` | 401 / 407 | no | yes |
| `broken` | other 4xx (404, 410, ...) | **yes** | yes |
| `server_error` | 5xx | **yes** | yes |
| `redirect_loop` | more than 10 redirects | **yes** | yes |
| `bad_redirect` | redirect without a valid `Location` | **yes** | yes |
| `dns_failure`, `timeout`, `network_error` | no HTTP answer | **yes** | no |
| `blocked_403`, `rate_limited_429`, `blocked_captcha` | the site refused us: we cannot tell whether the link works | unknown | no |
| `ssrf_blocked`, `credential_url_refused` | private / metadata address, or links with a user name or password in the URL's user-info part: never requested | unknown | no |
| `not_checked_robots`, `not_checked_robots_unreachable` | robots.txt disallows it / robots.txt failed | unknown | no |
| `not_checked_site_stopped`, `not_checked_page_cap`, `not_checked_budget`, `not_checked_time_limit`, `not_checked_external_off` | not requested | unknown | no |

Record statuses with no per-link or per-page charge (the per-run start fee still applies): `blocked_or_captcha` (the start page refused us), `start_unreachable`, `budget_reached`, `robots_disallowed`, `rate_limited`, `skipped_blocked`, `ssrf_blocked`, `invalid_input`.

### Pricing

**Pricing: $0.001 per link checked ($1 per 1,000 links) plus $0.0005 per crawled same-site page (each unique page once; a crawled page does not also count as a link), plus the start fee of $0.005 per run. Links and pages that could not be verified have no charge; the per-run start fee still applies.**

- Pay-per-event **`page-crawled`** ($0.0005): one event per unique page of your site (per start URL) that the Actor fetched to crawl it, the start page included, and that got an HTTP answer we can report: the verdicts marked "Charge: yes" above (so a crawled page that answers 404 or redirects counts). These are the report entries with `crawled: true` and `billedEvent: "page-crawled"`; `crawled: true` is also set on pages of your site the crawl reached but did not fetch (robots.txt, refused address, timeout), which have no `billedEvent` and no charge (the per-run start fee still applies).
- Pay-per-event **`link-checked`** ($0.001): one event per unique link (per start URL) checked without being crawled (external links, and links of your site beyond the depth limit) that got an HTTP answer we can report: the verdicts marked "Charge: yes" above.
- Each URL is charged at most once per start URL, as one of the two events (`billedEvent` in the report). Verdicts marked "Charge: no" have no charge, for crawled pages and links alike, and neither do the record statuses listed above; the per-run start fee still applies.
- Example: a run that crawls 200 pages of your site and checks 3,000 other links costs 200 x $0.0005 + 3,000 x $0.001 + $0.005 = $3.105.
- Events are charged only after the report is saved. A maximum total charge set on the run is respected: the price of the event is reserved before each request, and when the budget is used up the remaining links and pages are listed as `not_checked_budget`.
- A run may stop a fraction of a cent before your maximum charge, because the next link or page would not fit in the remaining budget.
- **What you pay even when every link fails:** the start fee of $0.005 per run. Apify charges it through its synthetic `apify-actor-start` event on every run: "Up to and including 1 GB of RAM, the event is charged once. Then, it's charged once for each extra GB of memory" ([Apify docs](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event#synthetic-start-event)). This Actor's memory is fixed at 1 GB, so the start fee is charged once per run.
- The synthetic `apify-default-dataset-item` event is not used (the Actor refuses to run if it is priced).

### Politeness and safety (kit defaults; may only be tightened)

- robots.txt is honoured for every request: crawled pages, link checks and every redirect hop, on your site and on external sites; `Crawl-delay` is respected.
- A robots.txt that returns 4xx (except 429) means no restrictions; a 429 is treated as a rate limit and stops the domain; 5xx or an unreachable robots.txt means nothing is fetched.
- If a site's host name does not resolve, its links are reported as `dns_failure`, not as robots.txt failures.
- At most about 1 request per second per site (scheme, host and port); fixed honest User-Agent `BrokenLinkCheckerActor/0.1.0 (+https://apify.com/luxe_tasks/broken-link-checker; honours robots.txt)`; no proxy rotation, no evasion.
- A 403, 429 (Retry-After over 60 s, or still 429 after one retry), a CAPTCHA/bot-wall page, or three 5xx answers stop that site for the rest of the run (and after a restart); its remaining links are listed as not checked and have no per-link charge.
- Private, loopback, link-local and cloud-metadata addresses are refused, re-checked after DNS and on every redirect. Links with a user name or password in the URL's user-info part are never requested.
- No logins, cookies, headers, proxies or session tokens can be supplied. robots.txt cannot be overridden in this version.

### Acceptable use

> **Acceptable use.** Only crawl sites you own or are authorized to audit; external links are checked with a single lightweight request (plus that site's robots.txt). By running this Actor you confirm that:
>
> 1. You will submit only start URLs of **sites you own or are authorized to audit**, and you will use the output lawfully.
> 2. You will **not** submit pages behind a login, paywall, or other access control, and the Actor does not accept passwords, cookies or session tokens.
> 3. You will **not** use it to get around CAPTCHAs, rate limits, IP blocks, or any other technical measure a site uses to restrict access.
> 4. You will comply with the target site's terms of use and robots.txt, and with all applicable law, including copyright and data-protection law (GDPR, CCPA and similar). You are the controller of any personal data in your results.
> 5. You will **not** use the Actor to harass, stalk, or profile individuals, or for vulnerability scanning or attack reconnaissance. The Actor reports link status, not vulnerabilities.
> 6. You will keep request volumes reasonable. The Actor enforces per-site rate limits and identifies itself in its User-Agent.
>    The Actor refuses private-network addresses and URLs with a user name or password, and skips URLs that robots.txt disallows. Report problems through the Actor's **Issues** tab.

### Privacy

This Actor stores no page content except link anchor text (up to 200 characters); reports hold URLs, HTTP statuses, redirect targets and that anchor text. The dataset and the report's start URL redact query strings; the report's link list keeps full URLs, including query strings. User names and passwords in a URL's user-info part (user:pass@host) are removed from every stored URL, including redirect targets. Secrets placed in a path or query string are not removed, except where query strings are redacted as described above. Do not submit start URLs that carry secrets or personal data in the query string.

> "Results are written only to your run's default Apify storage. The developer does not copy, view or retain them. If your results contain
> personal data, you are the controller and the developer acts only as your processor under Apify's Standard Actor Contract. Apify's retention settings apply."

### Third-party software

Runtime dependencies (apify, robots-parser, tldts) are listed in NOTICE with their licences.

### Runtime

Node 24 (`engines`, `.nvmrc`, Dockerfile `apify/actor-node:24`). Memory: fixed at 1024 MB (the work is network-bound and rate-limited to about 1 request per second per site,
so more memory does not make a run faster; the local peak was about 400 MB with three 5 MB pages holding 20,000 links each). Tests: `npm test`; end-to-end: `sh scripts/e2e-netns.sh` (local fixture sites only).

Contact: https://apify.com/luxe_tasks · Issues tab on the actor page.

# Actor input Schema

## `startUrls` (type: `array`):

One or more pages of a site you own or are authorized to audit. Each start URL gets its own report: its site (same registered domain) is crawled from there, and every link found is checked. With "Max link depth" 0 only the listed pages are opened and their links checked. URLs with a user name or password are refused; query strings are never logged. Don't include secrets or personal data in query strings.

## `maxPages` (type: `integer`):

Most same-site URLs requested in the whole run: pages crawled plus same-site links checked without crawling; the redirects of one URL count as that URL (default 500, at most 5,000). Links that are not checked are listed as not checked and have no per-link or per-page charge (the per-run start fee still applies).

## `maxDepth` (type: `integer`):

How many clicks from a start URL the crawl follows (0 = open only the start URLs; their links are still checked).

## `checkExternalLinks` (type: `boolean`):

Check links to other sites with one lightweight request each (HEAD, or a 1-byte GET if HEAD is not supported). Their redirects are reported (first hop and target), not followed. Other sites are never crawled.

## `timeoutSecs` (type: `integer`):

How long to wait for one response. A link that does not answer in time is reported as a timeout and has no per-link or per-page charge (the per-run start fee still applies).

## `maxConcurrency` (type: `integer`):

Items processed in parallel (each site still gets at most 1 request per second).

## Actor input object example

```json
{
  "startUrls": [
    "https://example.com/"
  ],
  "maxPages": 500,
  "maxDepth": 5,
  "checkExternalLinks": true,
  "timeoutSecs": 15,
  "maxConcurrency": 3
}
```

# Actor output Schema

## `results` (type: `string`):

Default dataset, one record per start URL, shown in the 'Results (one row per start URL)' view: index, url (query string redacted), status (success or the reason nothing was checked), startVerdict, pagesCrawled, uniqueLinks, linksChecked (= charged link-checked events), pagesCharged (= charged page-crawled events), brokenCount, redirectedCount, notCheckedCount, stoppedSites, verdicts (count per verdict), brokenLinks (first 100: url, status, verdict, foundOn, anchorText), charged, chargedEvents, chargedByEvent (charged events per event name, e.g. page-crawled and link-checked), error (why a start URL was not checked, or null), key and link of the full report. The 'Broken links' view has one row per broken link.

## `reports` (type: `string`):

Default key-value store, records starting with 'link-report-': one JSON report per start URL named link-report-<index>-<host>-<hash>-json with every unique link found: url, internal, crawled, verdict, broken, billed, billedEvent, status, method, redirectChain (each hop with status and Location), finalUrl, redirectNotFollowed, foundOn (up to 3 pages), occurrences, anchorText and error, plus the run's summary. The stored reports keep full URLs, including query strings.

## `runSummary` (type: `string`):

RUN_SUMMARY record: totals by status, charged events, chargedByEvent (total per event name since the last start or resume), chargeUncertainByEvent, itemCharges (per start URL saved since the last start or resume: chargeEvent, chargedEvents, chargedByEvent), budget reached and stopped domains for the run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://example.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("luxe_tasks/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://example.com/"] }

# Run the Actor and wait for it to finish
run = client.actor("luxe_tasks/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://example.com/"
  ]
}' |
apify call luxe_tasks/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,luxe_tasks/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mdvOgg6VhT6EDGU3A/builds/Eid7j3Slkn2JkcJcu/openapi.json