SEO & Broken Link Auditor

Apify Actor that crawls a website and produces a detailed SEO and link health report. Built with TypeScript, Crawlee CheerioCrawler, and the Apify SDK.

Features

• Site crawler – discovers pages from seed URLs with optional same-domain restriction
• Broken link detection – checks outbound links for 4xx/5xx errors and timeouts
• Redirect tracking – reports links that redirect to a different final URL
• SEO on-page audit per page:
  • brokenLinks[]
  • redirects[]
  • duplicateTitles
  • missingDescription
  • missingH1
  • canonicalIssues
  • sitemapUrl
  • robotsMeta
  • imagesMissingAlt[]
  • responseTimeMs
• Aggregated summary dataset items grouped by issue type
• Full JSON report saved to key-value store as OUTPUT
• Retries, rate limiting, and structured logging

Input

Field	Type	Default	Description
startUrls	array	https://crawlee.dev	Seed URLs to start crawling
maxPages	integer	50	Maximum pages to crawl
sameDomain	boolean	true	Only follow links on the same domain

Example input

{
    "startUrls": [{ "url": "https://example.com" }],
    "maxPages": 100,
    "sameDomain": true
}

See also ./examples/input.json.

Output

Per-page dataset items (recordType: "page")

{
    "recordType": "page",
    "url": "https://example.com/about",
    "title": "About Us",
    "statusCode": 200,
    "responseTimeMs": 342,
    "brokenLinks": [
        {
            "url": "https://example.com/old-page",
            "anchorText": "Old page",
            "statusCode": 404,
            "error": "HTTP 404"
        }
    ],
    "redirects": [
        {
            "url": "https://example.com/redirect-me",
            "anchorText": "Redirect",
            "statusCode": 301,
            "redirectChain": ["https://example.com/redirect-me"],
            "finalUrl": "https://example.com/new-page"
        }
    ],
    "duplicateTitles": [
        {
            "title": "Home",
            "urls": ["https://example.com/", "https://example.com/home"]
        }
    ],
    "missingDescription": false,
    "missingH1": false,
    "canonicalIssues": [],
    "sitemapUrl": "https://example.com/sitemap.xml",
    "robotsMeta": "index, follow",
    "imagesMissingAlt": [],
    "scrapedAt": "2026-07-05T12:00:00.000Z"
}

Summary dataset items (recordType: "summary")

Summary items are emitted for:

• overview
• broken-links
• redirects
• duplicate-titles
• missing-description
• missing-h1
• canonical-issues
• images-missing-alt

Full JSON report

The complete audit is also stored in the default key-value store under OUTPUT:

{
    "generatedAt": "2026-07-05T12:00:00.000Z",
    "pagesScanned": 25,
    "linksChecked": 412,
    "summary": {
        "totalBrokenLinks": 3,
        "totalRedirects": 8,
        "pagesMissingDescription": 2,
        "pagesMissingH1": 1,
        "pagesWithCanonicalIssues": 4,
        "totalImagesMissingAlt": 6,
        "duplicateTitleGroups": 1,
        "averageResponseTimeMs": 285
    },
    "pages": [],
    "summaries": []
}

Pricing

This Actor is designed for pay-per-result pricing at $0.005 per dataset item.

When publishing to Apify Store, configure the synthetic event:

• Event: apify-default-dataset-item
• Price: $0.005

Each page audit and summary item pushed to the default dataset counts as one result.

Local development

cd D:\Apify\factory\seo-broken-link-auditor
npm install
apify run

Run with the example input:

cp examples/input.json storage/key_value_stores/default/INPUT.json
apify run

Build TypeScript:

$npm run build

Run tests:

$npm test

Deploy

apify login
apify push

After pushing, enable monetization in Apify Console and set apify-default-dataset-item to $0.005.

Project structure

.actor/
├── actor.json
├── dataset_schema.json
├── input_schema.json
└── output_schema.json
examples/
└── input.json
src/
├── crawl-state.ts
├── link-checker.ts
├── main.ts
├── routes.ts
├── seo-analyzer.ts
├── types.ts
└── url-utils.ts

How it works

1. The crawler starts from startUrls and follows internal links up to maxPages.
2. Each page is parsed with Cheerio to extract SEO signals and outbound links.
3. Outbound links are checked with HTTP HEAD/GET requests (cached, rate-limited, retried).
4. robots.txt is scanned for sitemap URLs on the first crawled page.
5. After crawling, duplicate titles are computed across all audited pages.
6. Per-page results and summary items are saved to the dataset; the full report is saved to OUTPUT.