SEO Audit & Broken Link Checker: Site Crawler, 404s, Redirects

SEO Audit & Broken Link Checker: technical SEO site crawler for 404s, redirects and change alerts

Run a technical SEO site audit on a whole website and get every problem as data. The crawler works like an SEO spider and finds broken links, redirect chains, 4xx and 5xx pages, missing or duplicate titles and meta descriptions, noindex and canonical problems, orphan pages in the sitemap, and 37 checks in total. Each page gets its issues ranked by severity, each site gets a health score, and scheduled runs tell you what broke or got fixed since the last audit. Made for SEO agencies, web developers and site owners who audit many sites or need audits in a pipeline instead of a desktop app.

No browser, no login, no API key: the Actor reads public pages over plain HTTP, obeys robots.txt, and keeps a polite pace.

What you get

One row per page:

Field	Example	Meaning
url, statusCode	https://example.com/pricing, 200	The page and its HTTP status (redirects are rows too, with redirectTo and redirectChain)
issues	["broken-internal-links", "duplicate-title", "missing-h1"]	Every problem found, errors first
errors, warnings, notices	1, 1, 1	How many issues of each severity
indexable	true	200, HTML, not noindex and not canonicalized elsewhere
brokenLinks	[{ "url": "https://example.com/old", "status": 404, "anchor": "Old offer" }]	Links on this page that return 4xx/5xx or fail
linksToRedirects	[{ "url": "/team", "redirectsTo": "https://example.com/about/team" }]	Internal links you should update to their final URL
title, titleLength, metaDescription, metaDescriptionLength, h1		On-page basics
canonical, noindex, metaRobots, xRobotsTag, hreflang, lang		Indexing signals
depth, foundOn, inlinks, inSitemap	2, https://example.com/blog, 14, true	Clicks from the start page, where the crawler found the page, how many crawled pages link to it, and whether the sitemap lists it
wordCount, images, imagesWithoutAlt, structuredData, openGraph, viewport	840, 12, 3, ["Article", "BreadcrumbList"]	Content and markup
responseTimeMs, sizeKb	310, 48.2	Server response time and HTML size
newIssues, fixedIssues, alerts	["http-4xx"], ["missing-title"], ["Page now fails (404); it returned 200 last run."]	What changed since the last run of the same history

A SUMMARY record in the run's key-value store gives, per site, the health score (share of pages without errors), status codes, every issue type with the number of pages, examples and a short how-to-fix hint, all broken links with the pages that link to them, duplicate title groups, orphan pages, the sitemaps read, which search engines and AI crawlers (Googlebot, Bingbot, OAI-SearchBot, GPTBot, ClaudeBot, PerplexityBot and more) robots.txt lets read the site, and changes since the last run. A REPORT record in the same store is a ready-to-share HTML page (health score, issues ranked by severity with how-to-fix hints, broken links with the pages that link to them, duplicate titles, AI crawler access and changes since the last run): open it from the run's Output tab, then save it or send it to a client. The dataset has five views: Overview, Titles/descriptions/headings, Broken links and redirects, Indexing and sitemap, and Changes since last run.

Sample output

The home page row from a real run on 2026-11-12 against crawler-test.com, a site built with deliberate SEO errors for testing crawlers (lists shortened):

{
  "url": "https://crawler-test.com/",
  "statusCode": 200,
  "indexable": true,
  "issues": [
    "broken-internal-links",
    "broken-external-links",
    "duplicate-title",
    "missing-viewport",
    "mixed-content",
    "links-to-redirects",
    "meta-description-too-short",
    "missing-lang"
  ],
  "errors": 1,
  "warnings": 4,
  "notices": 3,
  "title": "Crawler Test Site",
  "titleLength": 17,
  "metaDescriptionLength": 40,
  "brokenLinkCount": 59,
  "brokenLinks": [
    {
      "url": "https://crawler-test.com/status_codes/status_404",
      "status": 404,
      "error": null,
      "anchor": "404 HTTP Status"
    }
  ],
  "linksToRedirects": [
    {
      "url": "https://crawler-test.com/redirects/redirect_1",
      "redirectsTo": "https://crawler-test.com/redirects/redirect_target"
    }
  ],
  "depth": 0,
  "inlinks": 164,
  "inSitemap": true,
  "responseTimeMs": 25
}

The 37 checks

• Errors: 4xx pages, 5xx pages, pages that could not be fetched, broken internal links, redirect loops, canonical pointing to an error page, missing title.
• Warnings: duplicate titles, titles over 60 or under 15 characters, multiple title tags, missing or duplicate meta descriptions, missing H1, images without alt, mixed content (HTTP resources on HTTPS pages), missing viewport, broken external links, redirect chains of 2+ hops, canonical pointing to a redirect, slow responses (over 1.5 s), orphan pages (in the sitemap but not linked), sitemap URLs that redirect or fail, noindex pages listed in the sitemap, invalid JSON-LD.
• Notices: long or short meta descriptions, multiple H1s, thin content (under 200 words), missing lang, internal links to redirects, noindex pages, canonicalized pages, HTML over 1 MB, pages more than 4 clicks deep, meta refresh redirects, hreflang without a self-reference.

The full list with descriptions is also in the SUMMARY record (issueTypes).

Use cases

• Technical SEO audit of a new client's site: sort by errors and fix the worst pages first.
• Broken link checker before and after a site migration or redesign.
• Weekly site health monitoring for agencies: one saved task per client on a schedule, and alerts tell you which pages started failing or became noindex.
• Sitemap QA: find sitemap URLs that redirect, return errors or are noindex, and pages that no link reaches.
• Content QA in CI: call the Actor from your pipeline after a deploy and fail the build if errors go up.

How to use it

1. Paste one or more websites (bare domains get https://).
2. Set Maximum pages per site (each audited page is one charged result).
3. Run it. For monitoring, add a weekly schedule: each run is compared with the previous one.

The crawler starts at your page, follows links on the same site (www. and non-www count as one site), then audits pages that only the XML sitemap knows about. Links to other sites, files such as PDFs, and internal pages beyond your page limit are checked with one lightweight request each (not charged).

Speed: requests to your site start at most about 6 per second, with up to 4 in flight, so 200 pages with their link checks usually take 1 to 4 minutes. A Crawl-delay in robots.txt is honored (up to 10 seconds), which makes the crawl slower.

Pricing

$0.005 per audited page ($5 per 1,000; $0.0045 on Apify's Silver plan, $0.004 on Gold and above; see the Pricing tab), plus Apify's standard start fee of $0.00005 per run. Link checks, the SUMMARY record and the change history are included with no per-link charge. Pages that could not be fetched at all (timeout, DNS or TLS error), pages behind a bot challenge or in a redirect loop, and start URLs that are not valid websites have no per-page charge. Set a maximum cost per run in Apify and the Actor audits only as many pages as that budget allows.

Automate it: schedules and integrations

• Weekly monitoring: in Apify Console, open Schedules, create a weekly schedule and add this Actor or a saved task. Use a different History name per client so their changes are tracked separately.
• Send results where your team works: under the Integrations tab you can connect Google Sheets, Slack, email, Make, Zapier, n8n or a webhook.
• API: start runs and download results with the Apify API or the official Python and JavaScript clients (see the API tab).
• AI agents (MCP): connect Claude, Cursor or any MCP client to Apify's MCP server at https://mcp.apify.com?tools=stellalytics/seo-site-audit.

Related Actors

• Bulk PageSpeed & Core Web Vitals Checker: Lighthouse scores and real-user Core Web Vitals for the pages this audit finds.
• Tech Stack & Domain Lookup: tech stack, DNS, email setup, SSL expiry and AI-crawler rules for a list of domains.
• AI Search Visibility Tracker: whether ChatGPT, Perplexity, Claude and Grok mention and cite your brand.

How it works, and what it does not do

• It identifies itself as SiteAuditBot with a link to this page, reads robots.txt on every host before requesting anything there, and obeys it. If robots.txt answers with a server error or 429, or cannot be reached at all, that host is treated as fully disallowed (RFC 9309) and nothing is crawled.
• It never logs in, never fills forms, and never works around CAPTCHAs or bot protection. A page behind a bot challenge is reported as such.
• It reads HTML as the server sends it and does not run JavaScript. Sites that build their content only in the browser show little text and few links (this is also how many crawlers see them).
• It only connects to public internet addresses and refuses private networks.
• It saves facts about pages (status, tags, counts and links), not copies of their content.

Limitations

• Up to 10,000 pages per site and 100 sites per run. Very large sites are best audited by section (start at a section URL and use Skip URLs containing for the rest).
• Pages only reachable through JavaScript, forms or a login are not found.
• Some servers refuse automated requests (401, 403, 429, or LinkedIn's 999), and some links time out. Those links are reported as unverified, not broken, and a page that times out or is rate-limited never raises a change alert. A link counts as broken only on a real 4xx/5xx answer, a domain that does not exist, a refused connection or an invalid certificate.
• Orphan pages are reported only when the crawl finished the whole site within the page limit, so a cut-short crawl does not flag false orphans.
• Changes in duplicate titles, duplicate descriptions and orphan pages are compared only when both runs covered the whole site, because a crawl cut short by the page limit may see a different set of pages.
• Title and description lengths are counted in characters, a common approximation of what search engines show.

FAQ

Can I audit a site I don't own? The Actor only reads public pages and obeys each site's robots.txt, like a search engine crawler. Auditing competitors' public pages is common SEO practice; respect the site's terms and don't audit at a pace the site can't handle (the defaults are gentle).

Is personal data collected? The Actor saves technical facts about pages, not page content. Titles, headings and link anchor texts are stored as they appear on the page, so a page that names a person in its title will show that title.

Why does a page show orphan-page? The sitemap lists it, but none of the crawled pages links to it, so visitors and search engines can only find it through the sitemap.

Why are some external links "unverified"? Sites such as LinkedIn answer automated requests with 999 or 403 even when the link works. Those are not counted as broken.

Is this affiliated with Google or any SEO tool? No.

Support

Open an issue on the Issues tab. Fixes for broken runs are a priority.