# SiteVerdict — Broken Link Checker (internal + external) (`siteverdict/siteverdict-broken-link-checker`) Actor

Crawl a site politely and check every internal and external link, image, script and stylesheet: 404s, dead domains, redirect chains and loops, mixed content and missing #anchors. Report grouped by source page, CSV/PDF export, monitoring and Slack alerts.

- **URL**: https://apify.com/siteverdict/siteverdict-broken-link-checker.md
- **Developed by:** [Aman Thakran](https://apify.com/siteverdict) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SiteVerdict — Broken Link Checker

**Find every broken link, image and script on your site, and see the exact page it sits on and the anchor text to fix.** Internal and external links are checked once each, politely, with per-host rate limits. You also get redirect chains and loops, mixed content and missing `#anchors`.

Who it is for: SEO agencies and site owners doing link hygiene, content teams after a migration, developers who want a pre-release check, and AI agents (MCP-ready).

### Three ways people use it

1. **Monthly link audit.** Crawl the whole site, then download `ISSUES-CSV` (one row per broken occurrence: source page, target, anchor text, status) and hand it to the editors.
2. **Migration QA.** After moving a site, find internal links that now redirect, redirect chains and 404s from old URLs.
3. **Scheduled monitor.** Run weekly with `monitoring` on (the default). A Slack/webhook alert fires only for **new** broken links or new active mixed content.

### Example output (sample mode, abridged)

```json
{ "type": "issue", "issue": "broken-link", "severity": "high", "sourcePage": "https://demo-links.siteverdict.test/",
  "target": "https://demo-links.siteverdict.test/missing-page", "status": 404, "anchorText": "Missing page",
  "fix": "Update the link to a working URL, remove it, or restore/redirect the missing page." }
{ "type": "issue", "issue": "missing-anchor", "target": "https://demo-links.siteverdict.test/guide#does-not-exist", "detail": "#does-not-exist not found on target" }
{ "type": "issue", "issue": "unverifiable-blocked", "severity": "info", "target": "https://partner.siteverdict.test/members", "status": 403 }
```

### What it checks

| Check | Notes |
|---|---|
| Broken links (4xx/5xx) | Internal and external `<a>`/`<area>` links. A HEAD failure is confirmed with GET before it is reported. |
| Dead domains, refused connections, TLS errors | Reported as broken, with the reason. |
| Broken resources | `<img>` + `srcset`, scripts, stylesheets, icons, iframes, video/audio/source, `<object>`, posters. |
| Redirect chains (2+ hops) and loops | Every hop is listed. |
| Internal links to redirecting URLs | So you can link to the final URL. |
| Missing `#anchors` | Same-page and cross-page fragments, checked against ids and `<a name>` on the target. SPA routes (`#!/`, `#/`) and text fragments are skipped. External fragments are optional. |
| Mixed content | Active (script, CSS, iframe: blocked by browsers) vs passive (images/media). |
| http:// links to your own https site | Insecure internal links. |
| Unverifiable (401/403/429/999) | Listed separately and **never counted as broken**. Many sites block automated checks, so check these manually. |

### Input highlights

`startUrls`, `crawl` (on by default), `maxPagesPerSite` (300, up to 10,000), `checkExternalLinks`, `checkResources`, `checkFragments`, `checkExternalFragments`, `maxLinksPerSite` (5,000), include/exclude patterns, `requestDelayMs` (500 ms for your site), `externalDelayMs` (1 s per external host), `monitoring`, `alertWebhookUrl` (secret), `sampleMode`.

### Output

- Dataset views: **Problems by source page** (`type: issue`), **Pages with problems** (`type: page`), **Link targets** (`type: link`, non-OK only unless `outputAllLinks`), **Summary**.
- Key-value store: `REPORT` (HTML grouped by source page), `REPORT-PDF`, `ISSUES-CSV`, `SUMMARY`.

### Pricing (pay per event)

| Event | Price | When |
|---|---|---|
| `page-scanned` | $0.0005 | Each HTML page fetched and scanned for links |
| `link-checked` | $0.0003 | Each unique link target checked. Targets skipped by robots.txt, targets refused as private/local/internal addresses (also on a redirect hop) and links on crawled pages are free; each link row shows `charged`. |
| `site-report` | $0.01 | Once per site: HTML/PDF/CSV report, monitoring diff and alert |

Examples: a 200-page site with 1,500 unique links ≈ **$0.56**. A 2,000-page site with 10,000 links ≈ **$4.01**. Nothing is charged in sample mode or for errors on our side. The run stops cleanly at your max-charge limit: the status message then starts with "Partial (max total charge reached)", each start URL that was not reached gets a free `error` row, and a site whose report did not fit in the budget has `reports: null` with a `reportsNote` (the report is not charged).

### FAQ

**Will it hammer other people's sites?** No. Each external host gets its own minimum spacing (1 s by default). Its robots.txt is fetched and obeyed, and retries back off. Links on hosts that disallow bots are reported as `robots-skipped` and are never requested.

**Why is a LinkedIn/Amazon link "unverifiable"?** Those sites answer automated requests with 999/403. We never try to get around that, so we mark them for manual checking instead of falsely calling them broken.

**JavaScript-rendered links?** Not followed. The checker reads the HTML your server sends, like most crawlers.

### Limits

- 10,000 pages and 50,000 unique link targets per site, 20 sites per run.
- External `#fragment` checks download the target page (up to 2 MB).
- Soft-404s on external sites (a 200 "not found" page) cannot be detected reliably and are not flagged.

### Sources & compliance

- Data: the public pages of the site you choose, and HTTP status responses of the URLs it links to. Fetched with the honest user agent `SiteVerdictBot/1.0 (+hello@webrevenuescan.com)`. robots.txt (RFC 9309) is obeyed on every host and every redirect hop. Checked 11 Oct 2026.
- Never bypasses CAPTCHAs, logins, paywalls or bot protection. No proxies.
- SSRF-safe: private, loopback and metadata addresses are refused, and every redirect hop is re-checked.
- No personal data is collected. Anchor text is stored only for links with problems. If your pages contain personal data, you are the controller of the audit (UK GDPR/PECR). Only check sites you own or may audit.
- The webhook URL is a secret input, redacted from logs and never output.

# Changelog

This Actor's version history is a separate document: https://apify.com/siteverdict/siteverdict-broken-link-checker/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

Start URL of each site (max 20). With crawling off, only these pages are scanned.

## `crawl` (type: `boolean`):

Follow internal links (and the sitemap) to scan every page. Turn off to check only the links on the start URLs.

## `maxPagesPerSite` (type: `integer`):

Upper limit of HTML pages scanned for links (1-10,000).

## `maxDepth` (type: `integer`):

How many clicks away from the start URL to follow.

## `useSitemap` (type: `boolean`):

Also scan pages listed in the sitemap.

## `checkExternalLinks` (type: `boolean`):

Check links to other websites (HEAD, with GET fallback). Each external host gets its own polite rate limit and its robots.txt is respected.

## `checkResources` (type: `boolean`):

Also check <img>, srcset, <script src>, stylesheets, icons, iframes and media files.

## `checkFragments` (type: `boolean`):

Verify that #fragment targets (ids) exist on internal target pages.

## `checkExternalFragments` (type: `boolean`):

Downloads external pages linked with a #fragment to verify the id exists (slower).

## `includeNofollowLinks` (type: `boolean`):

Check rel=nofollow/ugc/sponsored links too (they are still clickable by users).

## `maxLinksPerSite` (type: `integer`):

Cap on unique URLs checked per site (1-50,000).

## `includePatterns` (type: `array`):

Optional globs (/blog/\*\*) or /regex/ applied to pages to scan.

## `excludePatterns` (type: `array`):

Globs or /regex/ for pages not to scan.

## `includeSubdomains` (type: `boolean`):

Treat blog.example.com etc. as internal.

## `maxConcurrency` (type: `integer`):

Parallel lanes across different hosts (each host is still rate-limited).

## `requestDelayMs` (type: `integer`):

Polite spacing per host. robots.txt Crawl-delay is honoured when it is larger.

## `externalDelayMs` (type: `integer`):

Minimum spacing between requests to the same external host.

## `requestTimeoutSecs` (type: `integer`):

Per request.

## `outputAllLinks` (type: `boolean`):

Also push rows for links that are OK.

## `monitoring` (type: `boolean`):

Keeps a snapshot per site in the named store 'siteverdict-broken-link-history' and flags new and fixed problems.

## `projectName` (type: `string`):

Optional stable name used to store the monitoring snapshot, e.g. 'client-acme'. Defaults to the site host.

## `alertWebhookUrl` (type: `string`):

Optional. Slack incoming-webhook URL or any https:// endpoint that receives a JSON POST when monitoring finds new problems. Stored encrypted as an Apify secret, never logged or written to the output.

## `sampleMode` (type: `boolean`):

Run on a built-in demo with planted problems. No network requests and nothing charged. Use it to see the output format.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/"
    }
  ],
  "crawl": true,
  "maxPagesPerSite": 5,
  "maxDepth": 10,
  "useSitemap": true,
  "checkExternalLinks": true,
  "checkResources": true,
  "checkFragments": true,
  "checkExternalFragments": false,
  "includeNofollowLinks": true,
  "maxLinksPerSite": 5000,
  "includeSubdomains": false,
  "maxConcurrency": 4,
  "requestDelayMs": 500,
  "externalDelayMs": 1000,
  "requestTimeoutSecs": 15,
  "outputAllLinks": false,
  "monitoring": true,
  "sampleMode": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `files` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://books.toscrape.com/"
        }
    ],
    "maxPagesPerSite": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("siteverdict/siteverdict-broken-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://books.toscrape.com/" }],
    "maxPagesPerSite": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("siteverdict/siteverdict-broken-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/"
    }
  ],
  "maxPagesPerSite": 5
}' |
apify call siteverdict/siteverdict-broken-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,siteverdict/siteverdict-broken-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6TysuhYg9SvWvczvS/builds/d7pRAzkwzC5XdbgUt/openapi.json