# Website SEO Audit & Broken Link Checker (`appropriate_songbird/site-seo-audit-link-checker`) Actor

Crawl your own website politely (robots.txt always honoured) and get a per-page SEO audit: titles, meta descriptions, headings, canonicals, redirects, duplicates and broken links. Nothing stored after the run.

- **URL**: https://apify.com/appropriate_songbird/site-seo-audit-link-checker.md
- **Developed by:** [Puneet](https://apify.com/appropriate_songbird) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website SEO Audit & Broken Link Checker

**Crawl your own site politely and get a page-by-page SEO audit plus a list of broken links, as clean JSON, in one run.**

Point the Actor at the home page of a site you own (or are allowed to test). It follows internal links, optionally seeds the crawl from your XML sitemap, and checks each HTML page for the problems that cost rankings and clicks: missing or duplicate titles, weak meta descriptions, heading problems, canonical mistakes, noindex tags, images without alt text, thin pages, redirect chains, mixed content and broken links. You get one dataset row per page and a site-wide summary.

No browser, no login, no API key, no third-party SEO database. **robots.txt is always obeyed** and there is no switch to turn that off.

### What people use it for

- **Agencies and freelancers**: schedule a weekly run per client and diff the issue counts.
- **In-house SEO and web teams**: catch broken links and accidental `noindex` tags after a release.
- **Site migrations**: crawl before and after, compare redirect chains, titles and canonicals.
- **Content audits**: find thin pages, duplicate titles and duplicate descriptions in one table.
- **Developers**: call it from CI after a deploy and fail the pipeline when `brokenLinks` is not empty.

### What it checks (per page)

| Area | Issue codes |
|---|---|
| Status | `http-error`, `redirected`, `redirect-chain`, `slow-response` (over 3 s) |
| Title | `missing-title`, `title-too-long` (over 60), `title-too-short` (under 10), `duplicate-title` |
| Description | `missing-meta-description`, `meta-description-too-long` (over 160), `meta-description-too-short` (under 50), `duplicate-meta-description` |
| Headings | `missing-h1`, `multiple-h1` |
| Indexing | `noindex` (meta robots or X-Robots-Tag), `missing-canonical`, `canonical-elsewhere` |
| Page setup | `missing-lang`, `missing-viewport`, `missing-open-graph`, `invalid-json-ld` |
| Content | `thin-content` (under 150 words), `images-missing-alt` |
| Security | `mixed-content` (http resources on an https page) |
| Links | `broken-links` (4xx/5xx, DNS failure, timeout, connection refused) |

Each page also carries a simple `healthScore` (100 minus 20 per error and 5 per warning), the title, H1, description length, word count, canonical, language, structured-data types found (JSON-LD `@type`), link counts and the broken links found on that page.

### Sample output

```json
{
  "url": "https://example.org/pricing",
  "status": 200,
  "title": "Pricing",
  "titleLength": 7,
  "metaDescriptionLength": 0,
  "h1Count": 1,
  "wordCount": 212,
  "canonical": "https://example.org/pricing",
  "structuredDataTypes": ["Product"],
  "brokenLinkCount": 1,
  "brokenLinks": [{"url": "https://example.org/old-guide", "status": 404, "reason": "HTTP 404"}],
  "issues": ["title-too-short", "missing-meta-description", "broken-links"],
  "issueCount": 3,
  "healthScore": 75
}
```

The `SITE_SUMMARY` record in the key-value store lists issue counts by type and severity, duplicate titles, every broken link with the page it was found on, links that could not be verified, URLs skipped because of robots.txt and crawl notes.

### Inputs

| Input | What it does |
|---|---|
| Start URLs | 1 to 5 home pages. Only pages on the same host are crawled |
| Max pages | Pages fetched in total (1 to 2000, default 50) |
| Max link depth | Clicks from the start page (0 to 20, default 5) |
| Seed from sitemap.xml | Adds URLs from the sitemap(s) listed in robots.txt or at /sitemap.xml |
| Check links | HEAD-checks links that were not crawled, including external ones |
| Max extra link checks | Cap on those checks (0 to 3000, default 300) |
| Delay between page requests | 500 to 10000 ms. A higher Crawl-delay in robots.txt wins |
| Crawl URLs with query strings | Off by default, to avoid endless filter and sort variants |

Bad input (no URL, not an http(s) address, too many start URLs) stops the run at once with a plain message and no charge beyond the start event. If every start URL is unreachable or disallowed by robots.txt the run fails with the reason for each.

### Pricing

Pay per event: **$0.005 per run** plus **$0.003 per audited page**. A 100-page site costs about $0.31. Pages that return errors (404, 500, unreachable), URLs blocked by robots.txt and non-HTML files are free, and nothing is charged past the *Maximum cost per run* you set.

### Good to know

- **Polite by design.** One request at a time per host, at least 500 ms apart, a declared user agent (`SiteAuditLinkChecker`), retries with back-off on 429/5xx. External links are checked with HEAD only, at most once per second per host.
- **Access refusals are not "broken".** A 401, 403, 429 or 999 answer is listed as *unverifiable*. The Actor never retries around it, never solves captchas and never signs in.
- **Static HTML only.** Pages are read as the server delivers them, without running JavaScript. Content that only appears after scripts run is not seen.
- **Private networks are refused**, redirects included (localhost, internal IPs, cloud metadata addresses).
- **Pages larger than 3 MB** are truncated before parsing. Sitemaps ending in `.gz` are skipped.
- Link checks are best effort: some servers answer HEAD requests differently from browsers, and busy hosts can time out.

### Privacy and rights

Audit only sites you own or have permission to test. The Actor does not extract email addresses, phone numbers or any personal profile data, and it keeps nothing after the run: results live only in your own run's dataset and key-value store, plus run-health counters (runs started, succeeded, failed, last error) in a named store in your account.

Built on httpx, BeautifulSoup and lxml. This is an independent tool and is not affiliated with any search engine or SEO software vendor.

# Actor input Schema

## `startUrls` (type: `array`):

Home pages of the sites you own or have permission to audit (1 to 5). Only pages on the same host as each start URL are crawled. robots.txt is always obeyed.

## `maxPages` (type: `integer`):

Maximum number of pages fetched across all start URLs (1 to 2000).

## `maxDepth` (type: `integer`):

How many clicks from the start URL to follow (0 = only the start page, up to 20).

## `useSitemap` (type: `boolean`):

Also read the site's XML sitemap (found via robots.txt or /sitemap.xml) and add its URLs to the crawl queue.

## `checkLinks` (type: `boolean`):

Check the HTTP status of links found on the pages (internal links beyond the crawl limit and external links) and list the broken ones.

## `maxLinkChecks` (type: `integer`):

Cap on HEAD checks of links that were not crawled (0 to 3000). External hosts are contacted at most once per second.

## `requestDelayMs` (type: `integer`):

Pause between page fetches on the audited site (500 to 10000). A higher Crawl-delay in robots.txt wins.

## `includeQueryUrls` (type: `boolean`):

Off by default to avoid endless filter/sort/search variations. Turn on if your pages depend on ?parameters.

## Actor input object example

```json
{
  "startUrls": [
    "https://example.com/"
  ],
  "maxPages": 50,
  "maxDepth": 5,
  "useSitemap": true,
  "checkLinks": true,
  "maxLinkChecks": 300,
  "requestDelayMs": 500,
  "includeQueryUrls": false
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset with one item per crawled page: SEO fields, issues and broken links.

## `siteSummary` (type: `string`):

Issue counts per type, duplicate titles, broken links list and crawl notes.

## `runSummary` (type: `string`):

Run statistics and health counters.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://example.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("appropriate_songbird/site-seo-audit-link-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://example.com/"] }

# Run the Actor and wait for it to finish
run = client.actor("appropriate_songbird/site-seo-audit-link-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://example.com/"
  ]
}' |
apify call appropriate_songbird/site-seo-audit-link-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,appropriate_songbird/site-seo-audit-link-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/57QiaHYjro4dHU7Eh/builds/bIf0xUQnraigUwvGU/openapi.json