# Broken Link Checker API - 404s, Redirects & Retries (`stas_k/broken-link-checker`) Actor

Checkers that mark working links broken and cannot send your own headers sell a 403 from a bot-hating site as broken. Every check here retries timeouts, 5xx and 429, honours Retry-After, takes your User-Agent and Authorization headers, and blocked, private or invalid URLs cost nothing.

- **URL**: https://apify.com/stas_k/broken-link-checker.md
- **Developed by:** [Stas K](https://apify.com/stas_k) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 url checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker API - 404s, Redirects & Retries

Check any list of URLs - or a whole sitemap - for broken links in one run.
Every URL is tested with a real HTTP request and reported with its final
status code, redirect chain, response time, error type and how many retries
it took.

The most-installed broken link checker in the store carries a 2-star review
that says it plainly: links that are still working are scanned as broken,
and its own FAQ admits it does not support custom headers. This Actor is
built so that does not happen here:

| What usually happens | What happens here |
|---|---|
| One failed request marks a working link broken | Every check retries on network errors, timeouts, DNS failures, 429 and 5xx before the broken verdict - the row reports how many retries were used, and a Retry-After header sent by the site is respected |
| 403 from a bot-hating site is sold to you as a broken link | Custom headers and User-Agent are first-class input: pass your Authorization header and a real browser UA |
| Sitemap support means buying a second Actor | Pass sitemapUrl and all links inside it are extracted and checked together with your list |
| You are told "not blocked by robots.txt or a firewall" in troubleshooting | Blocked, private and invalid targets are reported honestly and cost nothing |

### Input

- **urls** - list of public HTTP(S) URLs or bare domains. Duplicates are
  checked once. Query strings are kept: two URLs that differ only by query
  are two checks.
- **sitemapUrl** - optional sitemap URL; all its links are checked together
  with the list. Sitemap index files are followed.
- **expectedStatusCodes** - a response is healthy only when its final status
  code is in this list. Empty means the default HTTP 200-399 rule.
- **headers** - extra request headers for every check (for example an
  Authorization bearer token). This is how you check endpoints that block
  the default bot User-Agent.
- **userAgent** - User-Agent header for the checks.
- **retries** - extra attempts on network errors, timeouts, DNS failures,
  429 and 5xx before a link is marked broken. Default 2. This is the core
  fix for false broken links: a link that answers 200 on a retry is reported
  healthy.
- **timeoutSec**, **maxRedirects**, **slowThresholdMs**, **maxItems**,
  **concurrency**, **cacheMinutes**, **timeBudgetSeconds** - self-explanatory
  caps, all visible in the Input tab.

### Output

One row per URL:

| field | meaning |
|---|---|
| url | the checked URL |
| statusCode | final HTTP status, null when the request never got an answer |
| broken | true when the final status is outside the expected rule, or the request failed after all retries |
| errorType | null, http_error, network_error, timeout, dns_error, tls_error, redirect_loop, invalid_input, private_target |
| error | human-readable reason |
| finalUrl | URL after following redirects |
| redirectCount | redirect hops followed |
| responseTimeMs | time of the accepted final response |
| slow | true when responseTimeMs exceeds slowThresholdMs |
| retriesUsed | how many retries this URL needed |
| expectedStatusCodes | the health rule applied to this row |
| source | list or sitemap |
| fromCache | true when a fresh verdict from another run was reused |
| checkedAt | UTC timestamp of the actual probe (kept on cache reuse) |
| cachedAt | UTC timestamp of the cache reuse, only on rows with fromCache true |

The RUN_SUMMARY key of the run's storage carries the totals: urlsChecked,
brokenCount, healthyCount, allHealthy, coverageComplete, cache hits,
invalid and private counts, rows refused by the spending limit, and rows
omitted by the time budget. allHealthy is true only when every listed URL
was checked and healthy - omitted, refused, dropped or failed rows make it
false, so a release gate can never pass on a half-checked list.

### Pricing

Pay per result: **$0.001 per checked URL** (charged as $1.00 / 1,000
url-checked events). Only URLs that were actually checked are charged.

- invalid and private targets are free, whether the private address is the
  link itself or the destination of its redirect chain;
- rows reused from the cache are free (a fresh verdict from another run
  inside the cache window costs nothing);
- a row refused by your spending limit is neither billed nor delivered.

You control the spending limit in the run options, so a huge sitemap can
never bill past what you allow.

#### Spending cap

If you do not set a spending cap, Apify applies its own default (about $4.63 per run). When that cap is reached the run stops early: the remaining items are skipped and you are charged only for what was actually delivered. For a full pass, set `maxTotalChargeUsd` in the run options.

# Actor input Schema

## `urls` (type: `array`):

Public HTTP(S) URLs or bare domains. Bare domains use HTTPS. Duplicate normalized URLs are checked once. Query strings are kept: two URLs that differ only by query are checked separately.

## `sitemapUrl` (type: `string`):

Optional sitemap URL. All links inside it are extracted and checked together with the urls list. Sitemap index files are followed.

## `expectedStatusCodes` (type: `array`):

A response is healthy only when its final status code is in this list. Leave empty for the default HTTP 200-399 rule.

## `headers` (type: `object`):

Extra headers sent with every check, for example an Authorization bearer token or an Accept header. This fixes the 403 block: many sites reject the default bot user agent, here you control it.

## `userAgent` (type: `string`):

User-Agent header for the checks.

## `timeoutSec` (type: `integer`):

Timeout per probe attempt.

## `retries` (type: `integer`):

Extra attempts on network errors, timeouts, DNS failures, 429 and 5xx responses before a link is marked broken. A link that answers 200 on a retry is reported healthy, not broken. This is the core fix for false broken links.

## `maxRedirects` (type: `integer`):

Redirect hops followed before the chain is reported as a redirect loop. Zero means redirects are not followed: the first response is the verdict.

## `slowThresholdMs` (type: `integer`):

Responses slower than this are flagged slow.

## `maxItems` (type: `integer`):

Input cap: only the first N unique URLs are checked. The coverage summary reports the rest.

## `concurrency` (type: `integer`):

How many URLs are checked in parallel.

## `cacheMinutes` (type: `integer`):

A fresh verdict from another run within this window is reused for free and marked fromCache. The cache key includes the expected status codes, headers and every probe setting (timeout, retries, redirects, slow threshold). Zero disables the cache.

## `timeBudgetSeconds` (type: `integer`):

Upper bound on the checking phase. When it is exhausted, the coverage summary reports omitted_by_time_budget.

## Actor input object example

```json
{
  "urls": [
    "https://example.com",
    "https://apify.com"
  ],
  "userAgent": "broken-link-checker/0.1 (Apify Actor)",
  "timeoutSec": 15,
  "retries": 2,
  "maxRedirects": 5,
  "slowThresholdMs": 2000,
  "maxItems": 10000,
  "concurrency": 8,
  "cacheMinutes": 60,
  "timeBudgetSeconds": 3300
}
```

# Actor output Schema

## `dataset` (type: `string`):

One row per URL

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com",
        "https://apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("stas_k/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://example.com",
        "https://apify.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("stas_k/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com",
    "https://apify.com"
  ]
}' |
apify call stas_k/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stas_k/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ErkPmlu6kXslBcn85/builds/uBazeaucpDRtNc5xD/openapi.json