# Broken Link Checker & SEO Site Audit - 404s & Redirects (`rjmacarthy/broken-link-checker`) Actor
Crawl a website and check every internal and external link: 404s, 5xx, DNS errors, timeouts and redirect chains, with the page and anchor text. Plus an on-page SEO audit per page. Bot-blocked links are flagged, not false 'broken'. $0.003 per page.
- **URL**: https://apify.com/rjmacarthy/broken-link-checker.md
- **Developed by:** [rj](https://apify.com/rjmacarthy) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet
## Pricing
from $3.00 / 1,000 page crawled and auditeds
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event
## What's an Apify Actor?
An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.
For examples already wired to this Actor's own input schema, see the [API](#api) section below.
Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).
# README
## Broken Link Checker & SEO Site Audit - Find 404s, Redirects, Missing Tags
Crawl any website and check **every internal and external link** it contains: 404s, 5xx errors, DNS failures, timeouts and redirect chains, each reported with the **page it appears on and its anchor text**. Every crawled page also gets an **on-page SEO audit** (title, meta description, H1, canonical, noindex, word count, images without alt). You pay **$0.003 per page** and all links on that page are checked at no extra cost, which is about **15x cheaper** than per-link checkers on a typical page with 50 links. Links to big sites that block bots (LinkedIn, Amazon, Cloudflare captchas) are flagged as `unverified` instead of being reported as false "broken" links.
### What data can you get
| Item type | Fields |
|---|---|
| `broken-link` | `linkStatus` (broken / timeout / redirect), `statusCode`, `error` (ENOTFOUND, ECONNREFUSED, SSL_ERROR, timeout), `targetUrl`, `sourceUrl` (page it's on), `anchorText`, `linkType` (internal / external / image), `finalUrl`, `redirectChain` |
| `page` | `url`, `statusCode`, `finalUrl`, `redirectChain`, `responseTimeMs`, `depth`, `title` + length, `metaDescription` + length, `h1`, `h1Count`, `canonical`, `metaRobots`, `noindex`, `lang`, `wordCount`, `internalLinks`, `externalLinks`, `brokenLinksOnPage`, `redirectLinksOnPage`, `unverifiedLinksOnPage`, `imagesWithoutAlt`, `issues[]` |
| `link` (all-links mode) | Same fields as `broken-link`, for every checked link, including `ok`, `unverified` and `rate-limited` ones |
| `SUMMARY` record | Pages crawled, URLs checked, link counts by status, broken links by status code, top SEO issues, per-site results, requests and warnings |
**Link statuses:** `ok` · `broken` (4xx/5xx, DNS error, connection refused, SSL error) · `redirect` (internal link that goes through 3xx hops) · `unverified` (external site answered 401/403/999 or a captcha/Cloudflare challenge, so it blocks bots and can't be verified) · `rate-limited` (429 even after a retry) · `timeout` (no answer in 8 s).
**SEO issue codes:** `missing-title`, `title-too-long` (>60 chars), `title-too-short` (<10), `missing-meta-description`, `meta-description-too-long` (>160), `missing-h1`, `multiple-h1`, `noindex`, `canonical-other-page`, `images-missing-alt`, `slow-response` (>3 s), `broken-links`, `redirect-chain` (the page was reached through a redirect, or links to internal URLs that redirect).
### Use cases
- **SEO agencies:** run a site audit for a new client and hand over a list of broken links with the exact page and anchor text to fix.
- **Site owners and marketers:** find 404s after a migration or redesign, and internal links that still point at old (redirecting) URLs.
- **Link-building teams:** find dead outbound links on a site (broken-link building outreach).
- **QA and developers:** check a staging site or docs section before release; schedule it after deploys.
- **Monitoring:** schedule it weekly with **Only new broken links** and get alerted only when something new breaks.
### How to use
1. Add one or more **Start URLs** (a homepage, or a section such as `https://example.com/blog/` combined with an include pattern).
2. Set **Max pages per site** (each page is one charged event).
3. Run. Open the **Broken links** view for the fix list and the **Page SEO audit** view for on-page issues.
The crawler stays on the start URL's host (`www.` is ignored), follows links breadth-first up to **Max crawl depth**, and checks every link it finds (HEAD first, GET if the server refuses HEAD). Each URL is checked once per run, even if hundreds of pages link to it, and every occurrence is reported.
#### Input examples
Quick audit of a small site:
```json
{
"startUrls": [{ "url": "https://books.toscrape.com/" }],
"maxPages": 10,
"checkExternalLinks": true
}
```
Only a documentation section, with broken images, every link listed:
```json
{
"startUrls": [{ "url": "https://crawlee.dev/js/docs/quick-start" }],
"includeUrlPatterns": ["https://crawlee.dev/js/docs/*"],
"maxPages": 200,
"checkImages": true,
"outputMode": "all-links"
}
```
Weekly monitor that only reports new problems:
```json
{
"startUrls": [{ "url": "https://quotes.toscrape.com/" }],
"maxPages": 500,
"onlyNew": true,
"excludeUrlPatterns": ["*/tag/*", "/\\?page=\\d+/"]
}
```
Patterns are globs where `*` matches anything (`https://example.com/blog/*`), or regular expressions wrapped in slashes (`/\/docs\//`). They decide which pages are **crawled**; links to other pages are still checked.
### Output example
Real output from a 3-page run on quotes.toscrape.com (the site's author links redirect with HTTP 308, so they show up as internal `redirect` links):
```json
[
{
"type": "page",
"id": "3379c189862fef41",
"url": "https://quotes.toscrape.com/",
"statusCode": 200,
"finalUrl": "https://quotes.toscrape.com/",
"redirectChain": [],
"responseTimeMs": 107,
"depth": 0,
"title": "Quotes to Scrape",
"titleLength": 16,
"metaDescription": null,
"metaDescriptionLength": 0,
"h1Count": 1,
"h1": "Quotes to Scrape",
"canonical": null,
"metaRobots": null,
"lang": "en",
"wordCount": 277,
"internalLinks": 47,
"externalLinks": 2,
"imagesWithoutAlt": 0,
"noindex": false,
"brokenLinksOnPage": 0,
"redirectLinksOnPage": 8,
"unverifiedLinksOnPage": 0,
"linksChecked": 49,
"issues": [
"missing-meta-description",
"redirect-chain"
],
"siteUrl": "https://quotes.toscrape.com/",
"scrapedAt": "2026-11-08T12:33:36.041Z"
},
{
"type": "broken-link",
"id": "37aea823408e082b",
"url": "https://quotes.toscrape.com/author/Albert-Einstein",
"sourceUrl": "https://quotes.toscrape.com/",
"targetUrl": "https://quotes.toscrape.com/author/Albert-Einstein",
"anchorText": "(about)",
"linkType": "internal",
"linkStatus": "redirect",
"statusCode": 200,
"error": null,
"finalUrl": "http://quotes.toscrape.com/author/Albert-Einstein/",
"redirectChain": [
{
"url": "https://quotes.toscrape.com/author/Albert-Einstein",
"statusCode": 308
}
],
"scrapedAt": "2026-11-08T12:33:36.041Z"
}
]
```
### Pricing
Pay per event, no monthly fee:
| Event | Price | When |
|---|---|---|
| `page` | $0.003 | One crawled and audited HTML page, **including all of its link checks** (deduplicated across the crawl, max 100 external links per page by default) |
`broken-link` and `link` items are free. Pages that fail to load, non-HTML files and robots.txt-blocked URLs are not charged.
**Worked example:** a 200-page site costs 200 × $0.003 = **$0.60**, however many links it has. A per-link checker at $0.001 per link would charge about $10 for the same site at 50 links per page.
Set **Maximum cost per run** in the run options to cap spending: the Actor stops crawling cleanly when the budget is reached.
### Tips and limits
- **Start small** (`maxPages` 10-50) to see how many pages your site has; `maxPages: 0` means no limit.
- **Large menus cost time, not money:** a page with 300 internal links triggers up to 300 checks, but each URL is checked only once per run, so later pages are mostly cached and fast.
- **External links from datacenter IPs:** some big sites (LinkedIn, Amazon, Instagram, Stack Overflow) answer bots with 403/999 or a captcha. These are `unverified`, not `broken`; check them by hand if they matter.
- **No JavaScript rendering:** links that only exist after client-side JavaScript runs are not seen. Server-rendered and static sites (WordPress, Shopify, Docusaurus, Next.js SSR) work well.
- **robots.txt** is respected for crawling by default; single link checks to disallowed URLs (one HEAD request each) are still made so you know whether they work.
- `url` on link items is the link target; the page to fix is `sourceUrl`.
- Concurrency defaults to 3 parallel requests per host, with an 8 s timeout and a retry for 5xx / 429 / connection resets.
- Page and link items share one dataset. Each view shows its own columns but lists every item, so filter or sort by `type` when you export.
### FAQ
**Why is a link "unverified" instead of "broken"?**
The external site blocked the request as a bot (HTTP 401, 403, 999 or a Cloudflare/captcha page). The link probably works for humans. Reporting these as broken would flood your report with false positives.
**Does a redirect count as broken?**
No. Internal links that redirect are reported with `linkStatus: "redirect"` and the full chain, because updating them to the final URL saves a hop and crawl budget. Turn off **Report internal redirects** to treat them as `ok`. External redirects are always `ok`.
**How does "Only new broken links" work?**
The Actor remembers, per site, which broken links it reported in the previous run (in a named key-value store, `broken-link-checker-state`). Each run outputs only links that were not broken last time; page items are still output. A link that gets fixed and breaks again is reported again.
**Can I use a proxy?**
Usually not needed. If your site blocks the run's IP, enable Apify Proxy (datacenter). Residential proxies are not supported: if selected, the Actor falls back to Apify datacenter proxy with a warning.
**Is it legal?**
The Actor only requests public pages, like a browser or a search engine crawler would, and respects robots.txt by default. It does not log in or collect personal data. You are responsible for having the right to crawl the sites you audit and for complying with their terms and applicable law.
### Integrations
- **API:** start runs and fetch results with the [Apify API](https://docs.apify.com/api/v2) or the JavaScript / Python clients:
```bash
curl -X POST "https://api.apify.com/v2/acts/rjmacarthy~broken-link-checker/run-sync-get-dataset-items?token=YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://example.com/"}],"maxPages":50}'
```
- **Make, Zapier, n8n:** send new broken links to Slack, e-mail, Google Sheets or Jira.
- **MCP:** use it from Claude, Cursor or any MCP client through the [Apify MCP server](https://mcp.apify.com).
- **Schedules and webhooks:** run weekly and trigger an alert when the run finishes.
### Changelog
- **0.1** (2026-11): first release: link checking (HEAD/GET fallback, dedupe, unverified detection), redirect chains, on-page SEO audit, all-links mode, only-new monitoring.
### More scrapers by the same developer
- [AI Visibility Tracker: ChatGPT, Perplexity, Gemini, Claude](https://apify.com/rjmacarthy/ai-search-visibility-tracker)
- [AliExpress Scraper - Product Search, Prices, Ratings & Details](https://apify.com/rjmacarthy/aliexpress-scraper)
- [Apple App Store Scraper: Apps, Search & Reviews](https://apify.com/rjmacarthy/app-store-scraper)
- [AutoScout24 Scraper - Used Car Listings & Prices (Europe)](https://apify.com/rjmacarthy/autoscout24-scraper)
- [Bilibili Scraper - Videos, Search, Popular & Uploader Stats](https://apify.com/rjmacarthy/bilibili-scraper)
- [Bing Search Scraper: Organic Results, Ads & Related Searches](https://apify.com/rjmacarthy/bing-search-scraper)
Browse all my Actors on [my Apify Store profile](https://apify.com/rjmacarthy).
### Support
Found a bug or need a feature? Open an issue on the Actor's **Issues** tab and include the run ID.
# Actor input Schema
## `startUrls` (type: `array`):
Websites (or sections) to crawl, e.g. https://example.com/. The crawler stays on each start URL's host (www. ignored) and checks every link it finds.
## `maxPages` (type: `integer`):
Maximum pages to crawl and audit per start URL. Each page is one charged event; all links on it are checked at no extra cost. 0 = no limit (careful: big sites have thousands of pages).
## `maxDepth` (type: `integer`):
How many clicks away from the start URL to crawl. 0 = audit the start URL only (its links are still checked).
## `checkExternalLinks` (type: `boolean`):
Also check links pointing to other websites. External sites that block bots (403, 999, captcha) are reported as 'unverified', not 'broken'.
## `checkImages` (type: `boolean`):
Also check ![]()
URLs for broken images.
## `includeUrlPatterns` (type: `array`):
Only crawl pages whose URL matches one of these. Glob with \* (e.g. https://example.com/blog/\*) or a regex in slashes (e.g. //docs//). Links to other pages are still checked.
## `excludeUrlPatterns` (type: `array`):
Never crawl pages matching these (glob with \* or /regex/), e.g. */tag/* or /?page=\d+/. Links to them are still checked.
## `respectRobotsTxt` (type: `boolean`):
Do not crawl pages that robots.txt disallows. Single link checks (one HEAD request) are still made.
## `followRedirectsReport` (type: `boolean`):
Report internal links that redirect (with the full 3xx chain) as linkStatus 'redirect'. Turn off to treat redirects that end in 200 as 'ok'.
## `outputMode` (type: `string`):
broken-only: one page item per page plus one item per broken / timed-out / redirecting link. all-links: one page item per page plus one item per link checked (any status).
## `onlyNew` (type: `boolean`):
Monitoring mode: output only broken links that were not reported in the previous run for the same site (page items are still output). Schedule the Actor to get alerts on new broken links.
## `maxExternalLinksPerPage` (type: `integer`):
Cap on external links checked per page (link farms, footers). 0 = no limit.
## `maxConcurrency` (type: `integer`):
Parallel requests to any one host. Keep it low to be polite to the site.
## `proxyConfiguration` (type: `object`):
Usually not needed. Enable Apify Proxy (datacenter) if your site blocks the run's IP. Residential proxies are not supported: if selected, the Actor falls back to Apify datacenter proxy.
## Actor input object example
```json
{
"startUrls": [
{
"url": "https://books.toscrape.com/"
}
],
"maxPages": 10,
"maxDepth": 5,
"checkExternalLinks": true,
"checkImages": false,
"includeUrlPatterns": [],
"excludeUrlPatterns": [],
"respectRobotsTxt": true,
"followRedirectsReport": true,
"outputMode": "broken-only",
"onlyNew": false,
"maxExternalLinksPerPage": 100,
"maxConcurrency": 3,
"proxyConfiguration": {
"useApifyProxy": false
}
}
```
# Actor output Schema
## `brokenLinks` (type: `string`):
Broken / timed-out / redirecting links with source page and anchor text.
## `pages` (type: `string`):
One row per crawled page with title, meta description, H1, canonical, noindex, word count and issues.
## `allItems` (type: `string`):
Every item with every field.
## `summary` (type: `string`):
Pages crawled, links checked, broken links by status, top SEO issues, requests and warnings.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
{
"url": "https://books.toscrape.com/"
}
],
"maxPages": 10,
"checkExternalLinks": true,
"proxyConfiguration": {
"useApifyProxy": false
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("rjmacarthy/broken-link-checker").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": [{ "url": "https://books.toscrape.com/" }],
"maxPages": 10,
"checkExternalLinks": True,
"proxyConfiguration": { "useApifyProxy": False },
}
# Run the Actor and wait for it to finish
run = client.actor("rjmacarthy/broken-link-checker").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
{
"url": "https://books.toscrape.com/"
}
],
"maxPages": 10,
"checkExternalLinks": true,
"proxyConfiguration": {
"useApifyProxy": false
}
}' |
apify call rjmacarthy/broken-link-checker --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,rjmacarthy/broken-link-checker"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/H5rRvAH1LI5LqmEJT/builds/8kmH1OkRRtXgBKLyW/openapi.json