# Broken Link Checker: Bulk URL Status, 404 & Redirect Checker (`mosaicked_sponge/url-health-checker`) Actor

Find broken links, 404 errors and redirect chains across a whole website. Check sitemap URLs or URL lists in bulk: HTTP status code, redirect chain, response time and the page each broken link sits on, plus canonical, noindex and soft-404 flags.

- **URL**: https://apify.com/mosaicked_sponge/url-health-checker.md
- **Developed by:** [Jason](https://apify.com/mosaicked_sponge) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 1 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 url checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Broken Link & Sitemap Checker do?

![Broken Link Checker: Bulk URL Status, 404 & Redirect Checker - sample output](https://api.apify.com/v2/key-value-stores/QpfrAb6wndEDB0tDd/records/output-bil1UPiHMAtA2TXLB.png)

It checks every URL on your website and tells you what's broken, what redirects and what hurts your SEO. Give it **page URLs**, **sitemaps** (including sitemap index files and gzipped `.xml.gz` sitemaps) or just your homepage with **crawling** turned on, and you get one clean row per URL:

- **HTTP status** (200, 301, 404, 500 ...) and a plain-English `problem` label
- **Full redirect chain**: every hop with its status code and `Location`, plus the final URL and final status
- **Broken links with the page they were found on**, the anchor text, and how many pages link to them
- **Response time** in milliseconds, content type and size
- **SEO flags on your pages**: canonical URL and canonical mismatch, `noindex` (meta robots and `X-Robots-Tag`), mixed content (http:// images/scripts on https pages) and soft-404 hints
- **Redirect loops, DNS failures, SSL certificate errors, timeouts** are reported, never silently dropped

HTTP only, no browser, so it is fast and cheap: 500 URLs take about 10 seconds.

### Why use it?

- **Site migrations**: confirm every old URL 301s to the right new page in one hop, and find chains and loops.
- **SEO audits**: find 404s, soft 404s, `noindex` pages that are in your sitemap, and pages whose canonical points elsewhere.
- **Content QA**: find dead outbound links in your blog and see exactly which article to fix.
- **Monitoring**: schedule a weekly run and get alerted when something breaks.

### How much does it cost?

You pay **only for URLs actually checked**. No platform usage is billed to you.

| Apify plan | Price per 1,000 URLs checked |
|---|---|
| Free | **$0.75** |
| Starter (Bronze) | $0.75 |
| Scale (Silver) | $0.72 |
| Business (Gold) and up | $0.70 |

Plus $0.00005 per run start. Checking 1,000 URLs costs $0.75. Checking a 10,000-URL sitemap costs $7.50.

**Free, never charged:** duplicate URLs, input that is not a URL, the sitemap files themselves, sitemaps that fail to load, and URLs found above your `Max URLs to check` limit or your run's cost cap. The cost cap is respected before checking starts, so you are never charged for more than you allowed.

### How to use it

1. Paste your pages into **Start URLs** and/or your sitemap into **Sitemap URLs**.
2. Leave **Check links on pages** on to find broken links on those pages. Turn on **Crawl internal links** to discover more pages from your homepage.
3. Set **Max URLs to check** (default 1,000) to control cost.
4. Click **Start** and filter the results by `isBroken`, `problem` or `isRedirect`.

#### Example input

```json
{
  "startUrls": ["https://example.com/"],
  "sitemapUrls": ["https://example.com/sitemap_index.xml"],
  "crawlInternalLinks": true,
  "maxCrawlDepth": 2,
  "maxPagesToCrawl": 200,
  "maxUrls": 5000,
  "checkExternalLinks": true
}
```

### Input options

| Field | What it does | Default |
|---|---|---|
| `startUrls` | Pages to check. Links on them are checked too. URLs that look like sitemaps are read as sitemaps. | 2 demo URLs |
| `sitemapUrls` | XML sitemaps, sitemap indexes (nested), `.xml.gz`, or plain-text URL lists | none |
| `discoverSitemaps` | Read `robots.txt` of each start URL's site for `Sitemap:` lines (falls back to `/sitemap.xml`) | off |
| `maxUrls` | Hard cap on URLs checked and charged | 1,000 |
| `checkLinks` | Check every link on the start URL pages and crawled pages | on |
| `checkExternalLinks` | Also check links to other websites | on |
| `crawlInternalLinks` / `maxCrawlDepth` / `maxPagesToCrawl` | Discover more of your own pages by following internal links | off / 1 / 100 |
| `checkLinksOnSitemapPages` | Also check the links on every sitemap page (multiplies URL count) | off |
| `maxConcurrency` / `maxConcurrencyPerDomain` | Total parallel requests / politeness limit per host | 20 / 4 |
| `requestTimeoutSecs`, `maxRedirects`, `slowThresholdMs` | Timeouts, redirect limit, "slow" flag threshold | 15 s, 10, 3,000 ms |
| `userAgent`, `proxyConfiguration` | Custom user agent; optional Apify Proxy for sites that block datacenter IPs | honest UA, no proxy |

### Output

One row per unique URL. Example (a link that redirects twice):

```json
{
  "url": "https://crawler-test.com/redirects/redirect_chain_allowed",
  "statusCode": 200,
  "statusText": "OK",
  "finalUrl": "https://crawler-test.com/redirects/redirect_target",
  "isOk": true,
  "isBroken": false,
  "isRedirect": true,
  "redirectCount": 2,
  "redirectChain": [
    {"url": "https://crawler-test.com/redirects/redirect_chain_allowed", "status": 301, "location": "https://crawler-test.com/redirects/redirect_chain_disallowed"},
    {"url": "https://crawler-test.com/redirects/redirect_chain_disallowed", "status": 301, "location": "https://crawler-test.com/redirects/redirect_target"}
  ],
  "problem": "redirect_chain",
  "problems": ["redirect_chain"],
  "responseTimeMs": 62,
  "method": "GET",
  "urlType": "page",
  "source": "startUrl",
  "isInternal": true,
  "foundOn": null,
  "anchorText": null,
  "foundOnPages": [],
  "timesLinked": 0,
  "title": "Redirect Target",
  "canonicalUrl": null,
  "canonicalMismatch": false,
  "noindex": false,
  "mixedContentCount": 0,
  "softNotFoundHint": null,
  "checkedAt": "2026-10-20T17:05:40Z"
}
```

#### `problem` values

| Value | Meaning | `isBroken` |
|---|---|---|
| `ok` | 2xx, no issues | no |
| `redirect` / `redirect_chain` | 1 hop / 2+ hops before the final page | no |
| `not_found` | 404 or 410 | **yes** |
| `client_error` | other 4xx (400, 405, ...) | **yes** |
| `server_error` | 5xx | **yes** |
| `blocked` | 401, 403, 429 or 999: the site refused an automated request. Usually fine in a browser; try a proxy | no |
| `redirect_loop`, `too_many_redirects`, `bad_redirect` | Redirect problems | **yes** |
| `dns_error`, `connection_refused`, `connection_error`, `ssl_error`, `timeout` | No usable response | **yes** |
| `soft_404_hint` | 200 but the page says "not found", or a deep URL redirects to the homepage | no (review) |
| `noindex`, `canonical_mismatch`, `mixed_content`, `slow` | SEO / performance flags | no |

`problems` lists every flag on the row; `problem` is the most serious one.

#### Run summary (OUTPUT record)

Counts per problem, the **top broken links** (most-linked first, with the pages that link to them), the status of every sitemap file read, skipped input, and how many URLs were left unchecked by your limits.

### How it checks

- Pages (start URLs, sitemap URLs, crawled pages) are fetched with `GET` so their HTML can be analysed.
- Links are checked with `HEAD` first. If the server refuses `HEAD` or answers with an error, it retries with `GET` (many servers answer HEAD wrongly), so you don't get false 404s/405s. The `method` and `headFallback` fields show what happened.
- Redirects are followed manually, hop by hop, so the whole chain is recorded and loops are caught.
- Requests run in parallel, but never more than `maxConcurrencyPerDomain` at once to one host.
- `robots.txt` crawl rules are not applied to checks: this tool is meant for checking your own sites and the links on them.

### Limits

- No JavaScript rendering: links added by JavaScript after page load are not seen. Most sites put their links in the HTML.
- Some big sites (news, social networks) block datacenter IPs and answer `403`. Those rows are labelled `blocked`, not broken. Turn on the proxy option for them.
- Soft-404 detection is a hint (wording in the title/heading, or a redirect to the homepage), not a verdict.

### Other tools from this developer

- [Website Contact Extractor](https://apify.com/mosaicked_sponge/website-contact-extractor): emails, phones and social links from any website.
- [URL to Markdown](https://apify.com/mosaicked_sponge/url-to-markdown-for-llm-rag): turn pages or a whole sitemap into clean Markdown for AI.

### FAQ

**Does it support sitemap index files and .xml.gz?** Yes. Nested indexes are followed, gzipped sitemaps are unpacked, and plain-text URL lists work too. Each sitemap file's status is in the OUTPUT record.

**Will it hammer my server?** No. The default is at most 4 requests at a time per host (adjustable), with HEAD requests for links.

**How do I only get the broken ones?** Sort or filter the results table by `isBroken` or `problem`, or download only the rows you need in CSV/Excel/JSON. The OUTPUT record also lists the top broken links.

**Can I schedule it?** Yes, save the input as a task and add a schedule. Compare `brokenCount` in OUTPUT between runs.

# Actor input Schema

## `startUrls` (type: `array`):

Pages to check. Each page is fetched, checked (status, redirects, canonical, noindex, mixed content, soft 404) and, with "Check links on pages" on, every link on it is checked too. URLs that look like sitemaps (.xml, .xml.gz, /sitemap) are read as sitemaps automatically.

## `sitemapUrls` (type: `array`):

XML sitemaps, sitemap index files (nested indexes are followed), gzipped .xml.gz sitemaps or plain-text URL lists. Every URL listed is checked. Sitemap files themselves are free; failed sitemaps are reported in the OUTPUT record.

## `discoverSitemaps` (type: `boolean`):

For each start URL's site, read robots.txt for Sitemap: lines (falls back to /sitemap.xml) and check every URL they list.

## `maxUrls` (type: `integer`):

Hard cap on unique URLs checked (and charged) in one run. URLs found above the cap are counted in OUTPUT but not checked or charged.

## `checkLinks` (type: `boolean`):

Extract every <a href> link from the start URL pages (and crawled pages) and check it. Each broken link row tells you the page it was found on, its anchor text and how many pages link to it.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other websites. Off = only links on your own site(s).

## `crawlInternalLinks` (type: `boolean`):

Follow links on your own site to discover more pages (up to the depth and page limits below) and check the links on those pages too.

## `maxCrawlDepth` (type: `integer`):

How many clicks away from the start URLs to crawl (1 = pages linked from the start URLs).

## `maxPagesToCrawl` (type: `integer`):

Maximum number of internal pages fetched and scanned for links (start URLs included). Links on them are still checked up to "Max URLs to check".

## `checkLinksOnSitemapPages` (type: `boolean`):

Off (default): sitemap URLs are checked themselves (status, redirects, SEO flags). On: links on each sitemap page are checked too. This can multiply the URL count, so keep an eye on "Max URLs to check".

## `maxConcurrency` (type: `integer`):

Total requests in flight at once.

## `maxConcurrencyPerDomain` (type: `integer`):

Politeness limit: never more than this many requests at once to the same host. Lower it for small or rate-limited sites.

## `requestTimeoutSecs` (type: `integer`):

A URL that does not answer within this time is reported as a timeout.

## `maxRedirects` (type: `integer`):

Longer chains are reported as too_many_redirects. Loops are detected and reported as redirect_loop.

## `slowThresholdMs` (type: `integer`):

Responses slower than this get the 'slow' flag.

## `userAgent` (type: `string`):

Leave empty for the default, honest URLHealthChecker user agent.

## `proxyConfiguration` (type: `object`):

Off by default (direct, fastest, accurate timings). Turn on only if a site blocks the checker.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal",
    "https://crawler-test.com/redirects/redirect_chain_allowed"
  ],
  "discoverSitemaps": false,
  "maxUrls": 1000,
  "checkLinks": true,
  "checkExternalLinks": true,
  "crawlInternalLinks": false,
  "maxCrawlDepth": 1,
  "maxPagesToCrawl": 100,
  "checkLinksOnSitemapPages": false,
  "maxConcurrency": 20,
  "maxConcurrencyPerDomain": 4,
  "requestTimeoutSecs": 15,
  "maxRedirects": 10,
  "slowThresholdMs": 3000,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

One row per unique URL checked.

## `summary` (type: `string`):

OUTPUT record: counts, top broken links, sitemap status, skipped input.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/links/broken_links_internal",
        "https://crawler-test.com/redirects/redirect_chain_allowed"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("mosaicked_sponge/url-health-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        "https://crawler-test.com/links/broken_links_internal",
        "https://crawler-test.com/redirects/redirect_chain_allowed",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("mosaicked_sponge/url-health-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal",
    "https://crawler-test.com/redirects/redirect_chain_allowed"
  ]
}' |
apify call mosaicked_sponge/url-health-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mosaicked_sponge/url-health-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bil1UPiHMAtA2TXLB/builds/xrDLR4pNbz8uIt1RK/openapi.json