SEO Audit Crawler – Broken Links, Meta Tags, Redirects

Give it your website. It crawls the site and runs a full technical SEO audit with a broken link check in one run. You get one row per page with every issue found, one row per external link checked, and a site summary with a 0–100 SEO score and the top 10 fixes.

• Broken link checker: finds broken internal and external links (404, 410, 5xx, DNS and timeout errors) and lists the pages that link to them, so you know exactly where to fix them.
• Redirect checker: records every hop of each redirect chain and flags chains longer than one hop, redirect loops and internal links that point to redirects.
• On-page SEO: title and meta description (missing, too long, too short, duplicate), H1 count, canonical tag (missing or pointing elsewhere), noindex (meta robots and X-Robots-Tag), hreflang, <html lang>, Open Graph, missing image alt text, thin content, mixed content on HTTPS, JSON-LD schema types, response time and page size.
• Sitemap checks: reads sitemap.xml from robots.txt (indexes and gzip included), finds orphan pages that are in the sitemap but not linked anywhere, sitemap URLs that redirect or fail, and noindex pages listed in the sitemap.
• Issues-only mode for scheduled monitoring: only pages with errors or warnings and only broken links are saved.
• Cheap: $8 per 1,000 audited pages. That is 5× cheaper than the most popular SEO audit Actor. Error pages (404, 5xx) cost nothing.
• Polite: HTTP only (no browser), robots.txt and Crawl-delay respected, 3 parallel requests per site by default, an honest User-Agent, and at most 1 request per second per external host.

What can I use it for?

• Website audit before a launch or migration: catch broken links, redirect chains and missing tags before Google does.
• Monthly SEO health check: schedule the Actor with outputMode: issuesOnly and get only the new problems.
• Agencies and freelancers: produce a client site audit with a score, issue counts and the top fixes in minutes.
• Content teams: find duplicate titles and descriptions, thin pages and pages missing H1 or alt text.
• Link rot monitoring: find outgoing links to dead pages on blogs, docs and resource pages.
• AI agents: let an agent audit a site and propose fixes (see below).

Input

Field	Description
startUrls	One URL per site. The crawl stays on the same host (www and non-www count as the same site).
maxPages	Max HTML pages audited per site (default 50, max 10,000). Breadth-first, so the most important pages come first.
checkExternalLinks	Check outgoing links (HEAD, GET fallback, 8 s timeout). Default on.
maxExternalLinks	Max unique external URLs checked per site (default 500). The most-linked ones are checked first.
useSitemap	Use sitemap.xml to find more pages and run the sitemap checks. Default on.
outputMode	allPages (default) or issuesOnly
respectRobots, maxConcurrency, requestTimeoutSecs, userAgent	Advanced

{
  "startUrls": [{ "url": "https://www.example.com" }],
  "maxPages": 200,
  "checkExternalLinks": true,
  "maxExternalLinks": 500,
  "useSitemap": true,
  "outputMode": "allPages"
}

Output

The Output tab has four tables: Issues (one row per issue: URL, severity, code, details), Pages, External links and the Summary. The summary is also saved in the key-value store as SUMMARY.

Sample summary (crawler-test.com, a site built with SEO problems on purpose, 40 pages):

Site	Score	Grade	Pages	Errors	Warnings	Notices	Broken internal URLs	Broken external links
crawler-test.com	71	C	40	8	60	132	5	2
books.toscrape.com	83	B	10	10	12	20	0	0

Page row (shortened):

{
  "type": "page",
  "url": "https://crawler-test.com/links/broken_links_internal",
  "statusCode": 200,
  "redirectChain": [],
  "responseTimeMs": 163,
  "title": "Broken Links Internal",
  "titleLength": 21,
  "metaDescriptionLength": 40,
  "h1Count": 1,
  "wordCount": 30,
  "canonicalStatus": "missing",
  "indexable": true,
  "jsonLdTypes": [],
  "inSitemap": false,
  "linkedFrom": ["https://crawler-test.com/"],
  "issues": [
    {
      "severity": "error",
      "code": "BROKEN_INTERNAL",
      "message": "Page links to internal URLs that return 4xx/5xx or fail",
      "details": {
        "count": 5,
        "links": [{ "url": "https://crawler-test.com/links/not_found/foo1", "statusCode": 404 }]
      }
    },
    { "severity": "warn", "code": "THIN_CONTENT", "message": "Fewer than 200 words of visible text", "details": { "wordCount": 30 } }
  ],
  "errorCount": 1,
  "warningCount": 1
}

External link row:

{
  "type": "externalLink",
  "url": "https://invalid.crawler-test.com/",
  "statusCode": null,
  "ok": false,
  "botBlocked": false,
  "error": "DNS resolution failed",
  "sourceCount": 1,
  "sourcePages": ["https://crawler-test.com/"]
}

The summary row has score, grade, issueCounts (pages per issue code), topFixes (code, severity, affected pages, example URLs), statusCodes, brokenInternalExamples, sitemapUrlCount, crawlComplete and stoppedReason. See SAMPLE_OUTPUT.json for full rows.

Issue codes

Severity	Codes
error	HTTP_4XX, HTTP_5XX, FETCH_FAILED, REDIRECT_LOOP, BROKEN_INTERNAL, BROKEN_EXTERNAL, TITLE_MISSING, MIXED_CONTENT
warn	META_DESC_MISSING, TITLE_TOO_LONG (>60), META_DESC_TOO_LONG (>160), H1_MISSING, MULTIPLE_H1, CANONICAL_TO_OTHER, NOINDEX, REDIRECT_CHAIN (>1 hop), THIN_CONTENT (<200 words), DUPLICATE_TITLE, DUPLICATE_META_DESC, IMG_MISSING_ALT, IN_SITEMAP_NON_200, NOINDEX_IN_SITEMAP, SLOW_RESPONSE (>3 s)
info	TITLE_TOO_SHORT, META_DESC_TOO_SHORT, CANONICAL_MISSING, INTERNAL_REDIRECT_LINK, IN_SITEMAP_NOT_LINKED, HREFLANG_NO_SELF, LANG_MISSING, OG_MISSING, LARGE_PAGE

Score: 100 minus a penalty for each issue code. Errors weigh 10, warnings 4 and notices 0.5, scaled by the share of pages affected. 90+ = A, 75+ = B, 60+ = C, 40+ = D.

Pricing

Pay per event: you only pay for what was audited.

Event	Price
Page audited (HTML page, status 200)	$0.008 ($8 per 1,000)
External link checked (unique URL)	$0.0003 ($0.30 per 1,000)
Site summary (one per site)	$0.01

Free: 404/5xx pages, redirects, non-HTML files and internal link status checks. Example: a 200-page site with 300 external links costs 200 × $0.008 + 300 × $0.0003 + $0.01 = $1.70.

Actor	Price per 1,000 pages
smart-digital/complete-seo-audit-tool	$40
automation-lab/website-lighthouse-seo-audit	$14
logiover/website-seo-audit-crawler	$10
This Actor	$8, plus broken external links with source pages, a summary score and an issues-only mode

Set Maximum cost per run in the run options to cap your spend. The Actor stops cleanly when the limit is reached and always keeps enough budget to deliver the site summary.

FAQ

Does it render JavaScript? No. It reads the HTML the server sends, which is what most technical SEO checks need, and is fast and cheap. Sites that build all content in the browser will show as thin content.

Will it crawl other websites? No. It only crawls the host you give it (www and non-www count as the same). External links get one HEAD request (or GET if HEAD is refused) to read their status.

Why is an external link marked botBlocked? Some sites (LinkedIn, G2, Cloudflare-protected pages) answer automated requests with 401, 403, 429 or 999. Those links usually work in a browser, so they are reported with ok: true, botBlocked: true and do not count as broken.

Does it respect robots.txt? Yes, by default. URLs disallowed for SEOAuditCrawler (or *) are skipped and Crawl-delay is honored. Turn respectRobots off only for your own site.

Can I monitor my site every week? Yes. Create a schedule with outputMode: issuesOnly and connect a Slack, email or webhook integration.

What does "orphan page" mean here? A URL that is in your sitemap but is not linked from any crawled page (IN_SITEMAP_NOT_LINKED). For a reliable result, set maxPages high enough to crawl the whole site (see crawlComplete in the summary).

Use with AI agents / Apify MCP

The Actor works as a tool in the Apify MCP server for Claude, ChatGPT, Cursor and other MCP clients. Ask something like "Audit example.com (100 pages) and list the top 10 SEO fixes", and the agent calls the Actor and reads the SUMMARY record and the Issues view. Through the API:

curl -X POST "https://api.apify.com/v2/acts/gazidev~seo-audit-crawler/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://www.example.com"}],"maxPages":100,"outputMode":"issuesOnly"}'

Related Actors

• Sitemap URL Extractor (gazidev/sitemap-url-extractor): every URL from a site's sitemaps, with an optional status check.
• Website to Markdown (gazidev/website-to-markdown): turn pages into clean Markdown for LLMs and RAG.