SEO Audit & Broken Link Checker (On-Page Site Crawler)

Crawl your website and get an on-page SEO audit for every page plus a broken link report — in one run, as a table you can export to CSV, Excel or JSON. Each page gets its title, meta description, H1, canonical, robots/noindex, hreflang, lang, JSON-LD types, Open Graph tags, images without alt, word count, internal/external links, redirect chain and a list of issues ranked by severity (error / warning / notice). A site summary tells you which problems affect the most pages.

USD 3 per 1,000 audited pages (USD 0.003 per page). Error pages, redirects, files and link checks are free. Try it free with an empty input.

Part of IntegriApps MicroApps. Support: Sunday to Friday.

What it does

• Finds your pages three ways: crawl internal links from your home page, read your sitemap.xml (sitemap index files included, found via robots.txt or /sitemap.xml), or audit only the URLs you list.
• Audits every HTML page for:
  • Title: missing, empty, too long (> 60 characters), too short (< 30), multiple tags, duplicate across pages.
  • Meta description: missing, too long (> 160), too short (< 70), multiple tags, duplicate across pages.
  • Headings: missing H1, multiple H1s, H2 count.
  • Indexing: canonical (missing, invalid, multiple, pointing to another URL), meta robots and X-Robots-Tag noindex / nofollow.
  • International: <html lang> and hreflang (invalid language codes, missing self-reference).
  • Structured data: JSON-LD present, @types found (including @graph), invalid JSON-LD blocks.
  • Social: Open Graph og:title, og:description, og:image, og:type and twitter:card.
  • Content: visible word count (thin pages), images without alt (with sample URLs; alt="" is accepted as decorative), meta viewport.
  • HTTP: status code, full redirect chain (each hop with its status), redirect loops, HTML response time.
• Broken link checker: every internal link on audited pages is checked (pages that were crawled reuse their result; other links such as PDFs or pages beyond your page limit get a HEAD request, then GET if the server refuses HEAD). External links are optional. You get each broken link with the page it was found on, its HTTP status or network error, and the anchor text. 401/403/429 answers are reported as restricted, not broken, to avoid false alarms.
• Site summary in the key-value store (OUTPUT): pages audited, status code counts, issues ranked by severity and number of affected pages (with example URLs), unique broken links with where they were found, noindex pages and average word count.
• Polite and safe crawler: identifiable User-Agent (IntegriAppsSeoAudit), always respects robots.txt (including Crawl-delay) for pages and link checks, retries with exponential backoff on 429/5xx (honoring Retry-After), timeouts, limited concurrency. Plain HTTP — no headless browser, no proxies, no CAPTCHA solving, no logins. Runs in 256 MB (a 200-page audit peaked at about 150 MB); for audits above ~2,000 pages set the run memory to 512 MB.

Use it on your own websites or on sites you have permission to audit (clients, employers). Pages disallowed by robots.txt are skipped and reported as blocked_by_robots_txt (free).

What it does not do

• It does not execute JavaScript: it audits the HTML your server sends (what most crawlers see first). Pages built only in the browser will show few words and missing tags — that is useful to know, but it is not a rendered audit.
• It does not measure Core Web Vitals / Lighthouse scores, keyword rankings, backlinks or search traffic.
• hreflang checks cover the codes and the self-reference on each page, not full return-link validation between pages.

Use cases

• Technical SEO audit of your site before or after a redesign, migration or CMS change.
• Find 404s and broken links (internal and external) and the exact pages that contain them.
• Fix duplicate titles and meta descriptions, missing H1s and missing alt text at scale.
• Check redirects after a migration: chains, loops and internal links that still point to old URLs.
• Monitor your site weekly with a schedule and compare the CSVs, or let an AI agent (via Apify MCP) audit a site and summarize the fixes.
• Agencies and freelancers: quick, cheap audits of client sites with a CSV deliverable.

How to use

1. Enter your home page in Website URLs (e.g. https://www.example.com/).
2. Pick How to find pages (crawl is the default) and Max pages to audit.
3. Optional: switch on Check external links, or limit the audit with Only URLs matching (e.g. /blog/).
4. Click Start. Open the Output tab: the SEO audit table has one row per page; the Broken links view has one row per broken link. Export as CSV/Excel/JSON, and open the OUTPUT record for the site summary.

Input

Field	Description	Default
startUrls	Website URL(s) to audit (empty = free 5-page demo on crawlee.dev)	—
mode	crawl, sitemap or urls	crawl
maxPages	Max pages fetched (1–5,000)	100
maxDepth	Crawl depth (clicks from the start URL)	5
includeUrlPattern / excludeUrlPattern	JavaScript regex filters for crawled URLs	—
checkInternalLinks	Check internal links for 404s / errors / redirects	true
maxInternalLinkChecks	Extra internal links checked (not crawled as pages)	500
checkExternalLinks	Also check links to other websites	false
maxExternalLinkChecks	Unique external URLs checked	300
maxConcurrency	Pages fetched in parallel (1–10)	3
linkCheckConcurrency	Link checks in parallel (1–10)	5

Example — audit up to 500 pages of the blog from the sitemap and check external links:

{
  "startUrls": [{ "url": "https://www.example.com/" }],
  "mode": "sitemap",
  "maxPages": 500,
  "includeUrlPattern": "/blog/",
  "checkExternalLinks": true
}

Output

One dataset row per URL fetched. status is ok (audited and charged), error (4xx/5xx, network error or redirect loop), redirect (redirects to a page audited separately), redirect_external, non_html or blocked_by_robots_txt — only ok rows are charged.

Real row from a run on crawler-test.com, a public test site built to exercise SEO crawlers (trimmed):

{
  "url": "https://crawler-test.com/robots_protocol/deepcrawl_excluded",
  "finalUrl": "https://crawler-test.com/robots_protocol/deepcrawl_excluded",
  "status": "ok",
  "httpStatus": 200,
  "errorsCount": 1,
  "warningsCount": 0,
  "noticesCount": 8,
  "issueCodes": "broken_internal_links, canonical_missing, lang_missing, low_word_count, meta_description_too_short, open_graph_incomplete, structured_data_missing, title_too_short, viewport_missing",
  "issues": [
    { "code": "broken_internal_links", "severity": "error", "message": "1 broken internal link(s)" },
    { "code": "canonical_missing", "severity": "notice", "message": "No rel=\"canonical\" link" },
    { "code": "meta_description_too_short", "severity": "notice", "message": "Meta description is 40 characters (recommended 70–160)" },
    { "code": "title_too_short", "severity": "notice", "message": "Title is 18 characters (recommended 30–60)" }
  ],
  "title": "Deepcrawl Excluded",
  "titleLength": 18,
  "metaDescription": "Default description Bl5Mm/8dZPlSqzsq0G4k",
  "metaDescriptionLength": 40,
  "h1": "Deepcrawl Excluded",
  "h1Count": 1,
  "canonicalUrl": null,
  "noindex": false,
  "lang": null,
  "hreflang": [],
  "jsonLdTypes": [],
  "ogTitle": null,
  "hasViewport": false,
  "wordCount": 12,
  "imagesWithoutAlt": 0,
  "internalLinks": 2,
  "externalLinks": 0,
  "brokenLinksCount": 1,
  "brokenLinks": [
    { "url": "https://crawler-test.com/robots_protocol/link_on_deepcrawl_excluded", "status": 404, "error": null, "internal": true, "anchorText": "link on deepcrawl excluded" }
  ],
  "redirectCount": 0,
  "redirectChain": [],
  "responseTimeMs": 14
}

Site summary (OUTPUT record) from the same 150-page run (trimmed):

{
  "mode": "crawl",
  "pagesFetched": 150,
  "pagesAudited": 124,
  "recordStatuses": { "ok": 124, "blocked_by_robots_txt": 7, "error": 7, "redirect": 10, "redirect_external": 2 },
  "statusCodes": { "200": 136, "301": 1, "302": 1, "404": 5 },
  "pagesWithErrors": 16,
  "pagesWithoutErrorsPercent": 89.3,
  "issuesBySeverity": { "error": 16, "warning": 73, "notice": 909 },
  "issues": [
    { "code": "broken_internal_links", "severity": "error", "pages": 7, "examples": ["https://crawler-test.com/", "https://crawler-test.com/redirects/url_redirect_chains"] },
    { "code": "http_4xx", "severity": "error", "pages": 5, "examples": ["https://crawler-test.com/redirects/redirect_to_404_target"] },
    { "code": "title_missing", "severity": "error", "pages": 2, "examples": ["https://crawler-test.com/titles/empty_title", "https://crawler-test.com/titles/missing_title"] }
  ],
  "internalLinksChecked": 317,
  "externalLinksChecked": 30,
  "brokenLinksCount": 87,
  "brokenLinks": [
    { "url": "https://crawler-test.com/robots_protocol/non_200_with_noindex", "status": 404, "error": "HTTP 404", "internal": true, "foundOn": ["https://crawler-test.com/"], "foundOnCount": 1 }
  ]
}

Issue codes

Severity	Codes
error	http_4xx, http_5xx, fetch_failed, redirect_loop, title_missing, broken_internal_links
warning	title_too_long, title_multiple, title_duplicate, meta_description_missing, meta_description_multiple, meta_description_duplicate, h1_missing, canonical_multiple, canonical_invalid, noindex, hreflang_invalid, jsonld_invalid, images_missing_alt, redirect_chain, broken_external_links
notice	title_too_short, meta_description_too_long, meta_description_too_short, h1_multiple, canonical_missing, canonical_points_elsewhere, nofollow_page, lang_missing, hreflang_no_self_reference, structured_data_missing, open_graph_incomplete, viewport_missing, low_word_count, redirected, internal_links_to_redirects, redirects_off_site, slow_response

Duplicates are checked only among indexable pages (no noindex, canonical pointing to itself or absent).

How much does it cost?

Pay per event: USD 0.003 per audited page (page-audited) — USD 3 per 1,000 pages. Only pages that return HTML successfully are charged. Not charged: 4xx/5xx pages, redirect-only URLs, non-HTML files, pages blocked by robots.txt, and all internal/external link checks. Platform usage is included in the event price.

• 100-page audit ≈ USD 0.30 · 1,000 pages ≈ USD 3.
• The empty-input demo (5 pages of crawlee.dev) is free.
• Set Max cost per run in the run options: the Actor audits only as many pages as fit in your limit and stops cleanly.

FAQ

Can I audit any website? Use it on websites you own or have permission to audit. The crawler identifies itself, respects robots.txt and Crawl-delay, keeps concurrency low and never bypasses logins, CAPTCHAs or blocks.

Why are some links "restricted" and not broken? Many sites answer automated requests with 401, 403 or 429 even when the page works in a browser. Those links are counted as restrictedLinks, not as broken, so your report has fewer false positives.

My site is built with React/Vue/Angular and shows few words. The Actor reads the HTML your server returns and does not run JavaScript. If your important content and tags only appear after JavaScript runs, search engines also need to render it — consider server-side rendering or prerendering.

How do I get a CSV? In the Output tab click Export and choose CSV or Excel. The SEO audit view gives one row per page; the Broken links view gives one row per broken link.

Why did the run stop before my page limit? Either the max cost per run was reached (see the log and limitReached in OUTPUT), the run timeout was near (timeBudgetReached; results so far are saved), or the crawl found no more internal pages.

Can I schedule it? Yes. Create a task with your input and add a schedule (e.g. weekly). Use webhooks or Zapier/Make/n8n to send the summary to Slack or email.

Does it follow nofollow links? In crawl mode, links with rel="nofollow" and pages with meta nofollow are not followed to discover new pages (their links are still checked).

Support

Open an issue on this Actor's Issues tab. Support: Sunday to Friday. Part of IntegriApps MicroApps by IntegriApps, SaaS for SMBs.