Website Health Checker — Tech Stack, Broken Links, SEO & Robots

Keywords: website health check, broken link checker, tech stack detector, on-page SEO audit, robots.txt validator, sitemap checker, website crawler audit.

One polite crawl: tech stack, broken links, redirect chains, robots.txt / sitemap checks and on-page SEO (title, meta, H1, canonical, alt text, noindex). For marketers, agencies and developers.

Why this Actor

• 50+ technology signals (CMS, frameworks, CDN, analytics, payments…)
• Internal + external broken links with source pages
• Redirect hops and loop detection
• robots.txt + XML sitemap validation
• Per-page SEO fields; respects robots.txt and Crawl-delay

Example input

{
  "startUrls": [
    { "url": "https://docs.python.org/3/" }
  ],
  "maxPagesPerSite": 10,
  "checkExternalLinks": true,
  "maxExternalLinks": 30,
  "concurrency": 5
}

Output

One summary row per site, then one page row per crawled URL.

{
  "type": "summary",
  "startUrl": "https://docs.python.org/3/",
  "pagesCrawled": 8,
  "robotsStatus": 200,
  "brokenLinkCount": 0,
  "techStack": [
    { "name": "Fastly", "category": "CDN" },
    { "name": "Sphinx", "category": "Static site generator" }
  ]
}

Pricing (pay per event)

• Actor start — $0.05
• Page / dataset item — $0.05 per result row (primary)

Tip: keep maxPagesPerSite small (5–10) for cheap first runs. A healthy site often returns brokenLinkCount: 0 — that is expected, not a no-op.

Limitations

No JavaScript rendering — client-only content/tech may be missed. Technology detection is heuristic.

Use cases & scheduling

• Broken link checker — find 4xx/5xx internal and external links with the page they appear on.
• Weekly website health monitoring — schedule weekly and compare brokenLinkCount / pagesWithSeoIssues over time.
• robots.txt & sitemap validation — status, crawl-delay and XML sitemap checks in one run.
• Tech stack lookup — detect CMS, frameworks, CDN and analytics on prospects' sites (agencies, sales research).
• On-page SEO audit — missing titles/meta descriptions/H1, canonical, image alt text, noindex.

Tip: create an Apify Schedule for the Actor or one of the tasks below to turn it into a recurring monitor, and add a webhook/integration to get results in Slack, email or Google Sheets.

Ready-made example tasks

One-click, pre-filled inputs you can run or copy:

• single-url-seo-health-check
• broken-link-checker-small-site
• robots-txt-sitemap-validator

FAQ

Does it render JavaScript?
No. It parses server HTML, so client-only content or tech can be missed.

Is the crawl polite?
Yes — it respects robots.txt and Crawl-delay and uses low concurrency by default.

How do I keep a run cheap?
Use maxPagesPerSite 1–10 and limit maxExternalLinks.

Why brokenLinkCount 0?
A healthy site often has no broken links — that is a valid result.