# Website Link Health - Broken Links & Recovery (`maydit/website-link-health-monitor`) Actor

Check links from public HTML pages with referring-page and anchor evidence, redirect chains, confirmed HTTP statuses and saved new/recovered issue tracking. No API key.

- **URL**: https://apify.com/maydit/website-link-health-monitor.md
- **Developed by:** [Brandt May](https://apify.com/maydit) (community)
- **Categories:** SEO tools, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.20 / 1,000 saved observations

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Link Health — Broken Links & Recovery

**Find the exact links that need attention, where they appear, and whether a previously broken target has recovered.**

Paste the pages that matter: a pricing page, campaign landing page, documentation index or migration checklist. This Actor reads their static HTML, checks each distinct target once, and returns the source page and anchor text beside the HTTP evidence. Save a baseline to see new issues, persistent failures and confirmed recoveries on later runs.

[Workflow guide](https://mayd-it.com/data-tools/actors/website-link-health-monitor/) · [More Mayd It data tools](https://mayd-it.com/data-tools/) · [Quality notes](https://mayd-it.com/data-tools/quality/)

### Start in three steps

1. Enter the exact **Pages to inspect**. The included public crawler-test page demonstrates five confirmed404links and one healthy link.
2. Keep **Check external links** enabled if outbound partners and citations matter. Images are optional.
3. Run, open **Link status and recovery**, and filter `linkStatus = broken`. Export JSON/CSV or open the readable run report.

```json
{
  "startUrls": ["https://crawler-test.com/links/broken_links_internal"],
  "checkExternal": false,
  "maxLinks": 20,
  "maxResults": 20
}
```

It checks the links on the supplied pages only. It does not recursively crawl the whole site. Add the important pages explicitly for a predictable scope and bill.

### Where it earns its place in your workflow

| Workflow | Useful evidence |
|---|---|
| Website maintenance retainers | New or persistent broken targets with exact referring pages and anchor text |
| Site migrations | Destination URLs and redirect chains for links on your checklist pages |
| Documentation quality | Dead references and recovery after a correction |
| Ecommerce campaign checks | Product and campaign links from landing pages; image `src` checks when enabled |
| AI retrieval pipelines | HTTP diagnostics before using discovered URLs in a downstream crawler |

### What a result looks like

This excerpt shows the shape returned by the included public fixture. Timestamps and source responses vary on each run.

```json
{
  "url": "https://crawler-test.com/links/not_found/foo1",
  "linkStatus": "broken",
  "httpStatus": 404,
  "headStatus": 404,
  "methods": ["HEAD", "GET"],
  "finalUrl": "https://crawler-test.com/links/not_found/foo1",
  "redirectChain": [],
  "foundOn": [{
    "sourcePage": "https://crawler-test.com/links/broken_links_internal",
    "anchorText": "Broken Internal Link 1",
    "kind": "link"
  }],
  "comparisonStatus": "not_requested",
  "issueChange": "not_requested"
}
```

One target can appear on several pages. Its `foundOn` array keeps distinct page/anchor observations, up to100per target; `referrersTruncated` is explicit if that limit is exceeded. `occurrences` counts those distinct observations, not duplicate identical HTML tags. Fragment-only links and `mailto:`, `tel:`, script, data and unsafe URLs are excluded. URL fragments are removed before deduplication; in-page anchors are not validated.

### Read the status correctly

| `linkStatus` | Meaning |
|---|---|
| `healthy` | Final HTTP response is2xx. This does not prove the page's content is correct. |
| `broken` | Confirmed404or410after the negative HEAD response was checked with GET. |
| `blocked` |401,403,429,451or999; the target may work for a person. No alternate route is tried. |
| `server_error` |5xxresponse; may be temporary and is not a confirmed dead link. |
| `client_error` |Another4xxresponse; review the recorded status. |
| `skipped_robots` / `robots_unavailable` |The site's crawl rules prevented inspection or could not be fetched reliably. |
| `timeout`, `unreachable`, `unsafe_target`, `redirect_limit` |The target could not be established within the documented request rules. |

A negative HEAD response is checked once with GET unless it is an explicit access refusal. Only response headers are needed for target status; a2xxsoft404body remains `healthy`. Redirect chains contain the URL, HTTP status and next destination of each followed hop, up to5hops. A redirect-limit failure is reported separately; it is not assigned an invented final destination.

### Saved changes without false recoveries

```json
{
  "startUrls": ["https://crawler-test.com/links/broken_links_internal"],
  "checkExternal": false,
  "comparePrevious": true,
  "snapshotName": "weekly-link-check",
  "maxLinks": 20,
  "maxResults": 20
}
```

Run the same watchlist again to compare it. `issueChange` reports:

- `new_issue`: a currently confirmed broken target without a broken prior observation.
- `persistent_issue`: the same target remains confirmed broken.
- `recovered`: a previously broken target now returns a2xxresponse.
- `baseline`, `changed` or `unchanged`: other recorded outcomes.

`comparisonStatus`, `changedFields`, `previousObservedAt` and `previous` retain the underlying comparison. Set `onlyChanges: true` to save only changed target snapshots. Referrer text is evidence, but changes to it alone do not count as an HTTP-status change.

History advances only after all selected pages and targets have been checked, outputs saved, and outcomes are determinate. A spending, request, result, time or discovery limit leaves the baseline unchanged. Blocked or unreachable checks also prevent baseline advancement. A target that disappears from the HTML is never marked recovered. Keep the same inputs/limits and baseline label, and avoid overlapping runs on one watchlist.

### Three input recipes

The first-run fixture and saved comparison recipe appear above. This second source exercises relative URL resolution:

```json
{
  "startUrls": ["https://crawler-test.com/links/relative_link/a/b"],
  "checkExternal": false,
  "maxLinks": 20,
  "maxResults": 20
}
```

All three exact recipes are available on the [website input recipes](https://mayd-it.com/data-tools/actors/website-link-health-monitor/#input-recipes).

### Pricing and cost control

Base pricing is **$0.002 per saved target result ($2per1,000)**, plus the configured Actor-start event. A saved result includes a diagnostic outcome such as blocked or unreachable. Each distinct target is charged at most once in a run even if several supplied pages link to it. Unchanged targets filtered by `onlyChanges` do not create charged dataset rows. See the Pricing tab for your plan's current rate.

Use `maxLinks`, `maxResults`, `maxRequests` and the Apify maximum-charge setting to bound work. The included six-target fixture normally saves six rows. Limits are independent: reaching one can produce partial results. The run's `SUMMARY` explains what was reached and whether history was updated.

### Outputs and integrations

- **Dataset:** one row per saved target, ready for JSON/CSV export.
- **SUMMARY:** page coverage, request counts, classifications, incomplete work and baseline decisions.
- **RUN_REPORT:** a self-contained readable report with links to the saved output.

Use Apify schedules for recurring checks and webhooks/integrations to route completed output. For AI agents, use the Actor's native Apify MCP tool or generated OpenAPI endpoint; the [website guide](https://mayd-it.com/data-tools/actors/website-link-health-monitor/) includes ready-to-copy API and MCP setup. An HTTP404result is useful audit data and does not itself mean the Actor failed.

### Scope and limits

No login, JavaScript execution, proxy, CAPTCHA workaround or form submission. Only public HTTP(S) on ports80/443; each DNS resolution and redirect is checked, and connections use the checked public addresses. Robots rules are respected on each origin, with at least500msbetween request starts to an origin. HTML bodies are capped at2MiB; robots bodies at512KiB; individual request deadlines are bounded. Up to20source pages,500unique targets and180seconds per run are supported.

At most10,000candidate tags per source page are scanned; a larger page reports incomplete discovery. Only `a[href]`, `area[href]` and optional `img[src]` are extracted. `srcset`, CSS backgrounds, scripts, stylesheets, JavaScript-added links, canonical tags and page semantics are outside this Actor's scope. Missing or changed source access is recorded rather than treated as a healthy result.

Need a different extraction source or a reproducible issue fixed? Open an Apify issue with the public URL, input and run link. Do not include credentials or private page content.

# Actor input Schema

## `startUrls` (type: `array`):

Public HTTP(S) page URLs. The Actor reads links on these exact pages; it does not crawl onward. Up to20URLs.

## `checkExternal` (type: `boolean`):

Check links pointing to other origins as well as links on the source site. External pages are never crawled.

## `includeImages` (type: `boolean`):

Also inspect image src URLs from the supplied HTML; srcset, CSS backgrounds, scripts and stylesheets are outside scope.

## `maxPages` (type: `integer`):

Read at most this many supplied URLs. Unreached input pages make the run incomplete and prevent baseline updates.

## `maxLinks` (type: `integer`):

Maximum unique discovered target URLs to check. If more are present, the run reports partial coverage and leaves saved history unchanged.

## `maxResults` (type: `integer`):

One charged row per unique checked URL. Lowering this below the discovered target count creates partial coverage.

## `comparePrevious` (type: `boolean`):

Save a complete determinate result set and compare later matching runs. First run establishes the baseline. Incomplete runs never advance it.

## `onlyChanges` (type: `boolean`):

Implies comparison. Skip targets whose HTTP classification, status, destination and redirect chain are unchanged. A missing link is never called recovered.

## `snapshotName` (type: `string`):

Use a stable label of1-80letters, numbers, underscores or hyphens for a watchlist; avoid overlapping writers.

## `maxRequests` (type: `integer`):

Hard ceiling covering source pages, robots.txt, redirects and target checks. Reaching the ceiling reports partial coverage.

## `maxRunSeconds` (type: `integer`):

Stop within this time budget, retaining saved results and explicit incomplete coverage. No automatic proxy or paid fallback.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal"
  ],
  "checkExternal": true,
  "includeImages": false,
  "maxPages": 5,
  "maxLinks": 100,
  "maxResults": 100,
  "comparePrevious": false,
  "onlyChanges": false,
  "snapshotName": "default",
  "maxRequests": 350,
  "maxRunSeconds": 90
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `report` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/links/broken_links_internal"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("maydit/website-link-health-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://crawler-test.com/links/broken_links_internal"] }

# Run the Actor and wait for it to finish
run = client.actor("maydit/website-link-health-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/links/broken_links_internal"
  ]
}' |
apify call maydit/website-link-health-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,maydit/website-link-health-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/alvSCjZTXtthrD6KR/builds/nIbmKHsPFqxg9uutX/openapi.json