SEO Audit and Broken Link Checker

Crawls a website from one or more start URLs and returns one row per page with its on-page SEO data, a list of SEO issues, and every broken link found on that page. Internal and external links are checked in the same run. Plain HTTP requests (no browser), so it is fast and cheap: 150 pages of books.toscrape.com took 13 seconds.

What it checks

Per page: HTTP status, redirect chain, response time, title and length, meta description and length, H1 texts, H2 count, canonical URL, meta robots and X-Robots-Tag, indexability, lang, viewport tag, word count, images and images without alt, internal/external link counts, hreflang tags, Open Graph title/description/image, JSON-LD and microdata types.

Links: every unique link on the audited pages is checked once. Internal pages are checked by crawling them; internal files (PDF, images, archives...) and external links get a HEAD request, with a GET fallback when the server rejects HEAD. A link counts as broken on HTTP 404, 410, any other 4xx except 401/403/429, any 5xx, or a connection error/timeout. Internal links that redirect are listed separately in redirectedLinks.

Issue codes (in issues):

Code	Meaning
fetch_failed, http_error	Page could not be fetched, or returned 4xx/5xx
missing_title, title_too_long, title_too_short, duplicate_title	Title missing, over 60 or under 10 characters, or shared with another audited page
missing_meta_description, meta_description_too_long, meta_description_too_short, duplicate_meta_description	Same for the meta description (limits 160 / 50)
missing_h1, multiple_h1	No H1, or more than one
missing_canonical, canonical_points_elsewhere	No canonical tag, or it points to another URL
noindex	Meta robots or X-Robots-Tag contains noindex
missing_lang, missing_viewport	No <html lang>, no viewport meta tag
images_missing_alt	At least one <img> without an alt attribute (alt="" is fine)
thin_content	Fewer than 200 words of visible text
redirected, slow_response	The page was reached through a redirect, or took over 2 s
broken_links, links_to_redirects	The page links to broken URLs, or to internal URLs that redirect

A site-wide summary is saved to the key-value store as SUMMARY: issue counts and every broken link with the pages it appears on (up to 20 per link).

Input

{
  "startUrls": [{ "url": "https://crawlee.dev" }],
  "maxPages": 20,
  "checkExternalLinks": true,
  "excludeUrlPatterns": ["/api/"],
  "maxConcurrency": 5
}

• startUrls: where to start. The crawler stays on the same host (www. and non-www count as one site). Several sites can be audited in one run.
• maxPages: stop after this many pages (default 100, max 10,000).
• checkExternalLinks: turn off to check only internal links.
• excludeUrlPatterns: regular expressions for URLs not to crawl (tag pages, filters, calendars).
• maxConcurrency: parallel page requests (default 5).

Example output

One dataset item, copied from a real run on crawlee.dev (the three broken links point to hostnames that do not exist, such as www.en.wikipedia.org):

{
  "url": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "finalUrl": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "statusCode": 200,
  "redirectChain": [],
  "depth": 2,
  "responseTimeMs": 73,
  "contentType": "text/html; charset=utf-8",
  "title": "Crawlee for Python v1 | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "titleLength": 79,
  "metaDescription": "Announcing the Crawlee for Python v1.0 release.",
  "metaDescriptionLength": 47,
  "h1": [
    "Crawlee for Python v1"
  ],
  "h2Count": 10,
  "canonicalUrl": "https://crawlee.dev/blog/crawlee-for-python-v1",
  "metaRobots": null,
  "isIndexable": true,
  "lang": "en",
  "hasViewport": true,
  "wordCount": 3644,
  "imagesCount": 12,
  "imagesMissingAlt": 0,
  "internalLinksCount": 72,
  "externalLinksCount": 28,
  "hreflang": [
    {
      "lang": "en",
      "url": "https://crawlee.dev/blog/crawlee-for-python-v1"
    }
  ],
  "openGraph": {
    "title": "Crawlee for Python v1 | Crawlee for JavaScript · Build reliable crawlers. Fast.",
    "description": "Announcing the Crawlee for Python v1.0 release.",
    "image": "https://crawlee.dev/img/crawlee-js-og.png"
  },
  "structuredDataTypes": [
    "BlogPosting"
  ],
  "brokenLinks": [
    {
      "url": "https://www.en.wikipedia.org/wiki/Robots.txt",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    },
    {
      "url": "https://www.developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    },
    {
      "url": "https://www.en.wikipedia.org/wiki/Device_fingerprint",
      "statusCode": null,
      "error": "ConnectError",
      "isInternal": false
    }
  ],
  "redirectedLinks": [],
  "issues": [
    "title_too_long",
    "meta_description_too_short",
    "broken_links"
  ],
  "issuesCount": 3,
  "error": null,
  "crawledAt": "2026-10-31T23:50:29+00:00"
}

Pricing

Pay per event: $0.002 per audited page ($2 per 1,000 pages). Link checks are included; there is no separate charge per link or per run start. Platform usage is included in the price. You can cap the cost of a run with the "Maximum cost per run" setting; the Actor then audits only as many pages as fit.

For comparison, on 2026-10-31 other SEO audit Actors on the Store charged $0.01–$0.04 per page, and a per-link broken-link checker charged $0.001 per link (a page with 50 links costs $0.05 there).

Use cases

• Find broken links and 404s before or after a site migration
• List pages with missing or duplicate titles and meta descriptions
• Check that important pages are indexable (no stray noindex, correct canonical)
• Schedule a weekly run and get alerted when brokenLinksCount in the summary goes up

FAQ

Does it respect robots.txt? Yes. URLs disallowed for SEOAuditBot or * are not crawled (their count is in the summary), and a Crawl-delay is honoured (up to 10 s), in which case pages are fetched one at a time. If robots.txt itself answers with a server error, the site is not crawled.

Why is a link marked broken that works in my browser? Connection errors and 5xx can also come from sites that block data-center traffic or a particular country. The statusCode and error fields tell you what happened; re-check such links by hand.

Why are links to LinkedIn or some shops not reported? They answer 403, 429 or 999 to automated requests. Those statuses are not counted as broken, because the link usually works for people.

Are internal pages beyond maxPages checked? No. Internal pages are checked by crawling them, so raise maxPages to cover the whole site. Internal files (PDF, images, downloads) are always checked.

Does it render JavaScript? No. It reads the HTML the server sends. Sites that build their content and links only in the browser show few pages and thin content.

Limits

• No JavaScript rendering, no login, no proxies, no anti-bot workarounds. Sites that block automated requests fail with a clear message.
• Up to 5,000 links that were not crawled as pages are checked per run.
• Results are written at the end of the run. A run that hits its timeout stops crawling early and still saves what it has.
• Core Web Vitals are not measured (that needs a real browser); responseTimeMs is the server response time.