# Broken Link Checker: crawl a site, find dead links (`examsens.ai/broken-link-checker`) Actor

Crawls your site to the depth and page limit you set, checks every internal and external link it finds, and reports the status, redirect chain, response time, source pages and anchor text of each one.

- **URL**: https://apify.com/examsens.ai/broken-link-checker.md
- **Developed by:** [Ed Rayyis](https://apify.com/examsens.ai) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.90 / 1,000 link checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Broken Link Checker: crawl a site, find dead links

Give it the address of a site and it crawls the site to the depth and page limit you set, checks every link it finds, and tells you which ones are broken, where each one appears and what anchor text points to it.

Each unique link is checked once, however many pages carry it. You get:

- The HTTP status and a category: `ok`, `redirected`, `clientError`, `forbidden` (401 and 403), `serverError`, `rateLimited` (429), `redirectError` (a 3xx with no usable Location), `redirectLoop`, `timeout`, `dnsError`, `connectionError`, `tlsError`, `invalidUrl` or `robotsDisallowed`
- A `broken` flag, true for 4xx (other than 429), 5xx, redirect faults, unknown hosts, refused connections, certificate failures and malformed URLs. Timeouts and 429 answers get their own category and are not flagged broken, so a slow or busy server does not read as a dead link
- The redirect chain with each hop's URL and status, the final URL and the redirect count
- Response time in milliseconds, a `slow` flag past your threshold, and the content type
- The first page that links to it with its anchor text, how many times it is linked, and up to 25 source pages, each with its anchor text, element and whether the link is `nofollow`
- Whether the link is internal or external, and the element it came from (`a`, `area`, `img`, `script`, `iframe` or `link`)

For an image link with no text, the anchor text is the image's alt text, marked `[image]`.

### How links are checked

Links are checked with a HEAD request. When HEAD fails or answers 400 or above, the link is checked again with GET, since some servers refuse HEAD. A request that times out on GET is tried once more. Pages the crawl opens are fetched with GET, and that answer is the page's result.

Pages on the start URL's host, with or without `www.`, are internal. Internal HTML pages are opened and their links collected, level by level, up to the crawl depth and page limit. External links are checked but never crawled. Links starting with `mailto:`, `tel:`, `javascript:` and `data:` are skipped, and the part after `#` is dropped.

### Input

- Sites to check (required). Each start URL is crawled as its own site.
- Maximum crawl depth (default 2). 0 checks only the links on the start page.
- Maximum pages to crawl (default 50) per start URL
- Maximum links per site (default 1,000)
- Check external links (default on)
- Check images, scripts, stylesheets and frames (default off)
- Only output problem links (default off). Leaves out links that answered 2xx, directly or after redirects.
- Follow robots.txt (default on). Internal URLs the site's robots.txt disallows are not requested and are listed with the category `robotsDisallowed`.
- Slow response threshold (default 3,000 ms), request timeout (default 20 seconds), parallel requests (default 10)
- User agent (optional). The default is `Mozilla/5.0 (compatible; BrokenLinkChecker/1.0)`.

### Output

Dataset: one item per unique link with `url`, `status`, `category`, `broken`, `linkType`, `element`, `sourcePage`, `anchorText`, `sourceCount`, `sources`, `finalUrl`, `redirectCount`, `redirectChain`, `responseTimeMs`, `slow`, `contentType`, `method`, `error` and `startUrl`. The start URL itself is an item with the link type `start`. Export as JSON, CSV or Excel, or read it over the API.

The key value store holds a `SUMMARY` record with, for each start URL, the pages crawled, links found, links checked, links left unchecked because a limit was reached, broken links and a count per category.

### Pricing

Pay per event: one event per link checked, including the pages the crawl opens. A link that "Only output problem links" leaves out of the dataset is still checked and counted. Links disallowed by robots.txt are not requested and not counted.

### Scope and conduct

It requests only public pages, sends no forms and does not log in. Email and phone links are skipped and never stored. Lower the parallel requests for small sites.

### Support

Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.

# Actor input Schema

## `startUrls` (type: `array`):

The page each crawl starts from, such as https://example.com. Each start URL is crawled as its own site. Pages on the same host, with or without www., count as internal.

## `maxDepth` (type: `integer`):

How many clicks from the start page to crawl. 0 checks only the links on the start page; 1 also crawls the internal pages it links to, and so on. Links found on the deepest pages are still checked.

## `maxPages` (type: `integer`):

Stop opening new internal pages for links after this many, per start URL.

## `maxLinksPerSite` (type: `integer`):

Stop collecting new unique links after this many, per start URL. Each link checked is one billable event.

## `checkExternalLinks` (type: `boolean`):

Also check links that point to other sites. External pages are checked but never crawled.

## `checkAssets` (type: `boolean`):

Also check the URLs in img, script, iframe, and stylesheet and icon link elements, not only hyperlinks.

## `onlyProblems` (type: `boolean`):

Leave out links that answered 2xx, directly or after redirects. Every link is still checked and billed.

## `respectRobotsTxt` (type: `boolean`):

Do not request internal pages that the site's robots.txt disallows. They are listed with the category robotsDisallowed and not billed.

## `slowThresholdMs` (type: `integer`):

Flag a link as slow when its response takes longer than this.

## `timeoutSecs` (type: `integer`):

How long to wait for each response. A link that times out is retried once with a GET request.

## `maxConcurrency` (type: `integer`):

How many requests run at the same time. Lower it for small or fragile sites.

## `userAgent` (type: `string`):

The User-Agent header sent with every request. Leave empty to send Mozilla/5.0 (compatible; BrokenLinkChecker/1.0).

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://books.toscrape.com"
    }
  ],
  "maxDepth": 2,
  "maxPages": 50,
  "maxLinksPerSite": 1000,
  "checkExternalLinks": true,
  "checkAssets": false,
  "onlyProblems": false,
  "respectRobotsTxt": true,
  "slowThresholdMs": 3000,
  "timeoutSecs": 20,
  "maxConcurrency": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://books.toscrape.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("examsens.ai/broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://books.toscrape.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("examsens.ai/broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://books.toscrape.com"
    }
  ]
}' |
apify call examsens.ai/broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,examsens.ai/broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/sZqV5XoSAAgqrXLnw/builds/jifU2JVazUsZZ2l0R/openapi.json