Website SEO Audit & Broken Link Checker (Site Crawler)

Website SEO Audit & Broken Link Checker

Crawl your own site politely and get a page-by-page SEO audit plus a list of broken links (404s), duplicate titles, missing meta descriptions and redirect chains, as clean JSON, in one run. From $0.003 per page, no subscription.

Try it in 10 seconds: press Start with the prefilled input. It audits one page and costs under one cent. Then replace the URL with your own site.

Point the Actor at the home page of a site you own (or are allowed to test). It follows internal links, optionally seeds the crawl from your XML sitemap, and checks each HTML page for the problems that cost rankings and clicks: missing or duplicate titles, weak meta descriptions, heading problems, canonical mistakes, noindex tags, images without alt text, thin pages, redirect chains, mixed content and broken links. You get one dataset row per page and a site-wide summary.

No browser, no login, no API key, no third-party SEO database. robots.txt is always obeyed and there is no switch to turn that off.

What people use it for

• Agencies and freelancers: schedule a weekly run per client and diff the issue counts.
• In-house SEO and web teams: catch broken links and accidental noindex tags after a release.
• Site migrations: crawl before and after, compare redirect chains, titles and canonicals.
• Content audits: find thin pages, duplicate titles and duplicate descriptions in one table.
• Developers: call it from CI after a deploy and fail the pipeline when brokenLinks is not empty.

What it checks (per page)

Area	Issue codes
Status	http-error, redirected, redirect-chain, slow-response (over 3 s)
Title	missing-title, title-too-long (over 60), title-too-short (under 10), duplicate-title
Description	missing-meta-description, meta-description-too-long (over 160), meta-description-too-short (under 50), duplicate-meta-description
Headings	missing-h1, multiple-h1
Indexing	noindex (meta robots or X-Robots-Tag), missing-canonical, canonical-elsewhere
Page setup	missing-lang, missing-viewport, missing-open-graph, invalid-json-ld
Content	thin-content (under 150 words), images-missing-alt
Security	mixed-content (http resources on an https page)
Links	broken-links (4xx/5xx, DNS failure, timeout, connection refused)

Each page also carries a simple healthScore (100 minus 20 per error and 5 per warning), the title, H1, description length, word count, canonical, language, structured-data types found (JSON-LD @type), link counts and the broken links found on that page.

Sample output

{
  "url": "https://example.org/pricing",
  "status": 200,
  "title": "Pricing",
  "titleLength": 7,
  "metaDescriptionLength": 0,
  "h1Count": 1,
  "wordCount": 212,
  "canonical": "https://example.org/pricing",
  "structuredDataTypes": ["Product"],
  "brokenLinkCount": 1,
  "brokenLinks": [{"url": "https://example.org/old-guide", "status": 404, "reason": "HTTP 404"}],
  "issues": ["title-too-short", "missing-meta-description", "broken-links"],
  "issueCount": 3,
  "healthScore": 75
}

The SITE_SUMMARY record in the key-value store lists issue counts by type and severity, duplicate titles, every broken link with the page it was found on, links that could not be verified, URLs skipped because of robots.txt and crawl notes.

Inputs

Input	What it does
Start URLs	1 to 5 home pages. Only pages on the same host are crawled
Max pages	Pages fetched in total (1 to 2000, default 50)
Max link depth	Clicks from the start page (0 to 20, default 5)
Seed from sitemap.xml	Adds URLs from the sitemap(s) listed in robots.txt or at /sitemap.xml
Check links	HEAD-checks links that were not crawled, including external ones
Max extra link checks	Cap on those checks (0 to 3000, default 300)
Delay between page requests	500 to 10000 ms. A higher Crawl-delay in robots.txt wins
Crawl URLs with query strings	Off by default, to avoid endless filter and sort variants

Bad input (no URL, not an http(s) address, too many start URLs) stops the run at once with a plain message and no charge beyond the start event. If every start URL is unreachable or disallowed by robots.txt the run fails with the reason for each.

Pricing

Pay per event: $0.005 per run plus $0.003 per audited page. Pages that return errors (404, 500, unreachable), URLs blocked by robots.txt and non-HTML files are free, and nothing is charged past the Maximum cost per run you set.

How much does it cost? Worked examples

Site	Pages audited	Cost
Landing page check	1	about $0.008
Small business site	25	about $0.08
Blog or store, default 50 pages	50	about $0.155
Larger site, 500 pages	500	about $1.51
Weekly run on a 100-page client site	100 per run	about $0.31 per run, $1.35 per month

Link checks (HEAD requests to links that were not crawled) are free. Set Maximum cost per run in the run options for a hard ceiling.

FAQ

Can I audit a competitor's site? Only sites you own or have permission to test. The Actor obeys robots.txt and crawls politely, but you are responsible for having the right to audit the site.

Does it find broken links on external sites? Yes. It checks outbound links with HEAD requests (up to Max extra link checks) and lists every 404, 410, 5xx, DNS failure and timeout together with the page that links to it.

How is it different from a browser extension or a desktop crawler? It runs in the cloud, can be scheduled (weekly, after every deploy) and returns JSON, CSV or Excel you can diff, chart or push to a webhook, Slack or Google Sheets through Apify integrations.

Why did a page not appear in the results? It was blocked by robots.txt, is a non-HTML file, is beyond Max pages or Max link depth, or has a query string while Crawl URLs with query strings is off. The SITE_SUMMARY record lists skipped URLs.

Does it run JavaScript? No, it reads server-delivered HTML, which is what search crawlers read first. Pages rendered only in the browser will look thin.

Can I fail my CI pipeline on broken links? Yes: run the Actor through the Apify API, then check brokenLinks in the summary.

Related Actors by the same developer: Sitemap, Robots.txt & Structured Data Validator for sitemap and schema.org checks.

Good to know

• Polite by design. One request at a time per host, at least 500 ms apart, a declared user agent (SiteAuditLinkChecker), retries with back-off on 429/5xx. External links are checked with HEAD only, at most once per second per host.
• Access refusals are not "broken". A 401, 403, 429 or 999 answer is listed as unverifiable. The Actor never retries around it, never solves captchas and never signs in.
• Static HTML only. Pages are read as the server delivers them, without running JavaScript. Content that only appears after scripts run is not seen.
• Private networks are refused, redirects included (localhost, internal IPs, cloud metadata addresses).
• Pages larger than 3 MB are truncated before parsing. Sitemaps ending in .gz are skipped.
• Link checks are best effort: some servers answer HEAD requests differently from browsers, and busy hosts can time out.

Privacy and rights

Audit only sites you own or have permission to test. The Actor does not extract email addresses, phone numbers or any personal profile data, and it keeps nothing after the run: results live only in your own run's dataset and key-value store, plus run-health counters (runs started, succeeded, failed, last error) in a named store in your account.

Built on httpx, BeautifulSoup and lxml. This is an independent tool and is not affiliated with any search engine or SEO software vendor.