# Site Health Auditor - Broken Links, Meta, Vitals, Weekly Diffs (`yasaslive/site-health-audit`) Actor

Crawls a site you own and returns one combined health report: broken internal and outbound links, redirect chains, meta/OG problems, Lighthouse Core Web Vitals, and a diff against the previous run.

- **URL**: https://apify.com/yasaslive/site-health-audit.md
- **Developed by:** [Eonix Pvt Ltd](https://apify.com/yasaslive) (community)
- **Categories:** Developer tools, Integrations, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.00005 / actor start

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Site Health Auditor — Broken Links, Meta, Vitals, Weekly Diffs

One crawl. One report. **And a diff against last week's run.**

Most site auditors hand you the same 400-line list of problems every time you run them, and you have no idea which ones are new. This Actor crawls a site you own, finds the problems, and then tells you the only thing that actually changes your Monday:

> **12 new issues, 5 resolved since 2026-08-08.**

Put it on a weekly Schedule, point it at Slack, and it becomes a standing early-warning system for your site.

> **This Actor is for auditing sites you own or operate.** It respects `robots.txt`, caps concurrency at 10, and checks third-party domains one request at a time. It is designed to be polite; please keep it pointed at your own properties.

***

### What it checks

| Area | What you get |
| --- | --- |
| **Broken links** | Internal links verified by the crawl itself; outbound links verified with `HEAD` (falling back to `GET`). Records source page, target, and status. |
| **Redirect chains** | Every redirect captured from the response history. One hop is a warning, two or more is an error. |
| **Metadata & Open Graph** | Missing `<title>`, duplicate `<title>` across pages, missing meta description, missing canonical, missing `og:image`. |
| **Accessibility & weight** | Images with no `alt` attribute, pages with multiple `<h1>`, HTML payloads over 2 MB. |
| **Core Web Vitals** | Lighthouse performance / SEO / accessibility scores plus LCP, CLS, INP and TBT on a smart sample of pages. |
| **Diff vs. last run** | New issues, resolved issues, and score movements per page — the reason to run this on a schedule. |

#### Who it's for

- **Solo founders and small teams** who own a marketing site or docs site and have no SEO tooling budget.
- **Agencies** managing a handful of client sites who want a weekly "what broke" digest per domain.
- **Engineering teams** who want a post-deploy check that content edits did not quietly break links or tank LCP.

***

### Three concrete use cases

**1. Weekly Monday-morning digest to Slack.**
Schedule the Actor for 07:00 every Monday with `slackWebhookUrl` set. You get one message per site: *"3 new issues, 8 resolved since last Monday"*, with the new ones listed. If nothing broke, the message says so and you move on.

**2. Post-migration link sweep.**
You moved a CMS, changed a URL structure, or merged two sites. Run once before the switch to capture a baseline, then again after. The diff is exactly your regression list — every internal link that used to work and now 404s, and every new redirect chain the migration introduced.

**3. Content-team guardrail.**
Writers publish weekly. Point the Actor at `/blog` and schedule it nightly. Missing meta descriptions, duplicate titles, and images without alt text surface within a day of publication instead of at the next quarterly audit.

***

### Input

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `startUrl` | string | **required** | The site to audit. The crawl stays on this origin. If the URL redirects (e.g. apex → `www`), the crawl follows it and audits the origin it lands on. |
| `maxPages` | integer | `300` | Hard cap on pages fetched. Max `2000`. |
| `maxOutboundChecks` | integer | `500` | Distinct off-site links to verify. Set to `0` to skip outbound checking. Max `5000`. |
| `runLighthouse` | boolean | `true` | Measure Core Web Vitals. Turn off for a much faster, cheaper link-and-metadata audit. |
| `lighthousePages` | integer | `5` | Pages to measure: homepage + most-linked pages, one per URL template. Max `20`. |
| `slackWebhookUrl` | string (secret) | – | Incoming-webhook URL. Receives the run summary and what changed. |
| `reportWebhookUrl` | string (secret) | – | Receives a JSON `POST` with the full summary and diff. |
| `proxyConfiguration` | proxy | – | Route traffic through Apify Proxy. Useful when a CDN rate-limits datacenter IPs. |

#### Minimal input

```json
{
  "startUrl": "https://your-site.com"
}
```

#### Fast link-only audit (no browser)

```json
{
  "startUrl": "https://your-site.com",
  "maxPages": 300,
  "maxOutboundChecks": 500,
  "runLighthouse": false
}
```

***

### Output

Every run produces:

1. **Dataset** — one record per issue, plus a final summary record.
2. **`REPORT.md`** in the key-value store — the human-readable report, leading with the diff.
3. **`REPORT.json`** in the key-value store — the same data, machine-readable.
4. **`PAGES.json`** in the key-value store — the raw per-page crawl data, if you want to do your own analysis.

All four are linked from the **Output** tab of every run, and the dataset ships three prebuilt views:

| View | Shows |
| --- | --- |
| **Issues overview** | Severity, type, the page to fix, the target, and whether the issue is new since the last run. |
| **Technical detail** | The same issues with the `details` payload flattened into columns — HTTP statuses, redirect hop counts, and fingerprints. |
| **Run summary** | Run-level totals: pages crawled, issue counts, new vs. resolved, and a direct link to `REPORT.md`. |

Every field is documented with a title, description and example in the dataset schema, so the column meanings show up in the Console rather than needing this README.

#### Sample output

All samples below are copied verbatim from a real local run against `https://quotes.toscrape.com` (`maxPages: 30`).

**Dataset — an issue record:**

```json
{
  "recordType": "issue",
  "type": "redirect_chain",
  "severity": "warning",
  "sourceUrl": "https://quotes.toscrape.com/author/Albert-Einstein",
  "targetUrl": "http://quotes.toscrape.com/author/Albert-Einstein/",
  "message": "Redirect: https://quotes.toscrape.com/author/Albert-Einstein → http://quotes.toscrape.com/author/Albert-Einstein/",
  "details": {
    "hops": 1,
    "chain": [
      "https://quotes.toscrape.com/author/Albert-Einstein",
      "http://quotes.toscrape.com/author/Albert-Einstein/"
    ],
    "finalStatus": 200
  },
  "fingerprint": "c2630c2595cac16a",
  "isNewSinceLastRun": true,
  "hostname": "quotes.toscrape.com",
  "runAt": "2026-08-15T16:31:14.498Z"
}
```

**Dataset — the final summary record:**

```json
{
  "recordType": "summary",
  "startUrl": "https://quotes.toscrape.com/",
  "origin": "https://quotes.toscrape.com",
  "hostname": "quotes.toscrape.com",
  "runAt": "2026-08-15T16:31:14.498Z",
  "pagesCrawled": 30,
  "pagesFailed": 0,
  "outboundLinksChecked": 4,
  "totalIssues": 126,
  "errorCount": 0,
  "warningCount": 126,
  "issueCountsByType": {
    "redirect_chain": 6,
    "duplicate_title": 30,
    "missing_meta_description": 30,
    "missing_canonical": 30,
    "missing_og_image": 30
  },
  "brokenLinkCount": 0,
  "lighthousePagesAudited": 0,
  "newIssueCount": 1,
  "resolvedIssueCount": 0,
  "isBaseline": false,
  "snapshotSaved": true,
  "robotsBlockedCount": 0,
  "newIssues": ["c2630c2595cac16a"],
  "resolvedIssues": []
}
```

**`REPORT.md` — the diff section on a repeat run:**

```markdown
### Since last run

Previous run: **2026-08-15 16:30:38 UTC**

#### New issues (1)

| Severity | Type | Page | Detail |
| --- | --- | --- | --- |
| 🟠 warning | Redirects | https://quotes.toscrape.com/author/Albert-Einstein | 1 hop → http://quotes.toscrape.com/author/Albert-Einstein/ |

#### Resolved since last run (0)
```

**`REPORT.md` — the Core Web Vitals table** (from a run with `runLighthouse: true`):

```markdown
| Page | Perf | SEO | A11y | LCP | CLS | INP | TBT |
| --- | --- | --- | --- | --- | --- | --- | --- |
| https://quotes.toscrape.com/ | 🟢 90 | 🟢 90 | 🟠 83 | 2.88 s | 0.000 | 16 ms | 0 ms |
| https://quotes.toscrape.com/login | 🟢 92 | 🟢 90 | 🟠 73 | 3.04 s | 0.000 | 16 ms | 0 ms |
```

***

### The weekly schedule pattern

This is what the Actor is built for.

1. Run it once against your site. That run is your **baseline** — no diff yet.
2. In the Apify Console, open the run and click **Schedule** (or **Schedules → Create new**).
3. Set a weekly cron, e.g. `0 7 * * 1` for Mondays at 07:00.
4. Add your `slackWebhookUrl` to the schedule's input.

From the second run on, every report opens with what changed. History is stored per hostname in a named key-value store called **`site-health-history`**, so you can schedule the same Actor against several sites and each keeps its own timeline.

To reset a site's history and start a fresh baseline, delete its key (named after the hostname) from that store.

***

### Pricing — pay per event

Mirror this table exactly in **Actor → Monetization → Pay per event** in the Apify Console.

| Event name | Title | Description | Price |
| --- | --- | --- | --- |
| `audit-page` | Page audited | One page of your site crawled and analysed — status code, redirect chain, title, meta description, canonical, Open Graph tags, headings, images missing alt text, page weight, and every internal and outbound link found on it. Charged once per page. | **$0.004** |
| `lighthouse-page` | Core Web Vitals page | A full Lighthouse audit of one page in a real Chromium browser, returning performance, SEO and accessibility scores plus LCP, CLS, INP and TBT. Charged once per page successfully measured. | **$0.05** |
| `report` | Health report | The combined report for the run — every issue found, grouped by severity, with the diff against your previous run and the Core Web Vitals table. Delivered as REPORT.md and REPORT.json. Charged once per completed run. | **$0.05** |

**When each event fires**

| Event | Fires |
| --- | --- |
| `audit-page` | Once per crawled page, after the crawl finishes and page data is durably stored. Pages that returned an error status are included — fetching them is the work, and their status is what the broken-link report is built from. |
| `lighthouse-page` | After each individual page audit succeeds. |
| `report` | After `REPORT.md` is written to the key-value store. |

Nothing is charged before the work it pays for is durably persisted. A run that fails mid-crawl bills nothing, and a Lighthouse page that errors is not charged.

Every charge bills **exactly one event** — the `count` parameter is never used. The platform can partially fulfil a bulk charge, which would silently put billing and delivered work out of step; charging one unit at a time means each call either bills one event or bills nothing, and the run stops charging the moment a limit is reached.

**Typical cost:** a 300-page site with 5 Lighthouse pages is `300 × $0.004 + 5 × $0.05 + $0.05` = **$1.50** per run, or about $6.50/month on a weekly schedule.

Because pages are billed individually, you pay for exactly what was crawled — a 40-page site costs `40 × $0.004` = **$0.16**, not a rounded-up block.

***

### Running it yourself

```bash
npm install
```

```bash
npm test
```

```bash
npm start
```

`npm start` compiles and runs against `storage/key_value_stores/default/INPUT.json`, which ships with a working sample. Results land in `storage/datasets/default/` and `storage/key_value_stores/default/`.

With the Apify CLI:

```bash
npx apify run
```

Everything at once — lint, type-check, tests, build:

```bash
npm run verify
```

***

### FAQ

**Does it respect robots.txt?**
Yes, always, and it cannot be turned off. URLs disallowed for `*` are skipped and counted in the report under "URLs skipped by robots.txt".

**Why does it report a redirect on a link that works fine?**
Because a redirect costs a round trip on every visit and dilutes link equity. One hop is reported as a *warning* — informational, fix it when convenient. Two or more hops is an *error*, because chains are usually accidental.

**Why do internal links to `http://` show up as outbound?**
"Same origin" means same scheme, host, and port. A link from `https://` to `http://` on the same host is genuinely cross-origin, and it is usually a bug worth seeing.

**A link works in my browser but is reported as broken.**
Check the `details.method` field on the issue. Some servers reject or ignore `HEAD`; the Actor already retries those with `GET` (including when `HEAD` hangs), and only reports the link broken if `GET` fails too, three attempts apart with exponential backoff. If `GET` genuinely fails from Apify's network but works from yours, the target is probably geo-blocking or rate-limiting datacenter IPs — try enabling Apify Proxy.

**Why did Lighthouse pick those pages?**
The homepage first, then the most-linked internal pages, taking only one page per URL template — `/blog/a` and `/blog/b` count as one template, so a 200-post blog contributes one representative post rather than crowding out the rest of the site.

**Why is INP always small or identical?**
A cold page load records no interactions, so lab INP is usually unavailable. The Actor falls back to Lighthouse's Max Potential FID, which is its stand-in for responsiveness. For real INP, use field data from CrUX.

**Can I audit a site I don't own?**
Please don't. The tool is deliberately polite rather than stealthy, and crawling someone else's site at volume is their call to make, not yours.

**Can it crawl a staging site behind a private IP or on localhost?**
No. URLs that resolve to loopback, private, or link-local addresses are refused — that guard is what stops a page full of `http://169.254.169.254/...` links from turning the crawler into an internal network probe. Audit staging over a public hostname instead.

**What happens on the very first run?**
It is the baseline. The report says so and skips the diff section. Every run after that compares against it.

***

### How it works

```
resolve startUrl ─▶ crawl (CheerioCrawler, robots.txt, maxConcurrency 10)
                        │
                        ├─▶ outbound link check (HEAD → GET, 1 req/domain at a time)
                        │
                        ├─▶ issue engine (pure functions, fully unit-tested)
                        │
                        ├─▶ Lighthouse (bundled Chromium over CDP, sequential)
                        │
                        └─▶ diff vs. site-health-history ─▶ dataset + REPORT.md/json ─▶ Slack/webhook
```

Issues are identified across runs by a stable fingerprint, `sha256(type|sourceUrl|targetUrl)`. It deliberately excludes counts and messages, so an `img_missing_alt` issue whose count goes from 3 to 4 stays *the same issue* rather than showing up as one resolved plus one new.

Plain HTTP is used for everything except Lighthouse — no headless browser is launched when `runLighthouse` is `false`.

***

### Changelog

#### 0.1.0 — 2026-08-15

- Initial release.
- Same-origin crawl with robots.txt support and a configurable page budget.
- Outbound link checking with `HEAD`→`GET` fallback, per-domain serialisation, and `Retry-After`-aware backoff.
- Ten issue types across links, redirects, metadata, Open Graph, accessibility, and page weight.
- Lighthouse Core Web Vitals on a template-deduplicated page sample.
- Run-to-run diff with new/resolved issues and per-page score deltas.
- `REPORT.md` and `REPORT.json` artifacts, Slack and generic webhook delivery.

***

### License

Apache-2.0. See [SECURITY.md](SECURITY.md) for the security model and [RUNBOOK.md](RUNBOOK.md) for operational procedures.

# Actor input Schema

## `startUrl` (type: `string`):

The site to audit. The crawl stays on this origin. If this URL redirects (for example example.com to www.example.com), the crawl follows it and audits the origin it lands on. Use this on sites you own.

## `maxPages` (type: `integer`):

Hard cap on pages fetched. The crawl stops here even if more pages are reachable.

## `maxOutboundChecks` (type: `integer`):

Distinct off-site links to verify. Outbound links are checked with HEAD (falling back to GET), one request at a time per domain. Set to 0 to skip outbound checking entirely.

## `runLighthouse` (type: `boolean`):

Measure Core Web Vitals on a sample of pages. Turn this off for a much faster, cheaper link-and-metadata-only audit.

## `lighthousePages` (type: `integer`):

How many pages to audit: the homepage plus the most-linked pages, with one representative page per URL template (so /blog/a and /blog/b count as one).

## `slackWebhookUrl` (type: `string`):

Optional. An incoming-webhook URL. The run summary, including what changed since the last run, is posted here when the audit finishes.

## `reportWebhookUrl` (type: `string`):

Optional. Receives a JSON POST with the full summary and diff when the audit finishes.

## `proxyConfiguration` (type: `object`):

Optional. Route crawl and link-check traffic through Apify Proxy. Useful when your own site or a CDN rate-limits datacenter IPs.

## Actor input object example

```json
{
  "startUrl": "https://example.com",
  "maxPages": 300,
  "maxOutboundChecks": 500,
  "runLighthouse": true,
  "lighthousePages": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `issues` (type: `string`):

One record per issue — severity, type, the page to fix, and whether it is new since the last run. The final record is a run-level summary.

## `report` (type: `string`):

The readable report. Opens with the diff against your previous run, then issues grouped by severity, then the Core Web Vitals table.

## `reportJson` (type: `string`):

The same audit as structured JSON: summary, diff, every issue, Lighthouse results, and per-page crawl stats. Use this to feed dashboards or alerting.

## `pages` (type: `string`):

Every crawled page with its status, redirect chain, metadata and link counts, for your own analysis.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://example.com"
};

// Run the Actor and wait for it to finish
const run = await client.actor("yasaslive/site-health-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrl": "https://example.com" }

# Run the Actor and wait for it to finish
run = client.actor("yasaslive/site-health-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://example.com"
}' |
apify call yasaslive/site-health-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,yasaslive/site-health-audit"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/goVScEUl8IZddRxE9/builds/plJZ0ERNfMH4TROma/openapi.json