SEO Audit Crawler: On-Page Checks for Every Page

Crawl a website and get an on-page SEO audit for every page: title and meta description length, H1 and heading structure, canonical, noindex, language, mobile viewport, image alt text, Open Graph and schema.org markup, with a plain-English list of issues per page and the duplicate titles across the site. You pay only for pages that were audited; broken, blocked and skipped pages are free.

Use cases

• Site owners and marketers: find the pages with a missing title, a missing H1 or no meta description in one run.
• Agencies and freelancers: a quick technical on-page check of a client site before a proposal or a launch.
• Before a launch or a seasonal sale: check new landing pages for noindex left on by mistake, wrong canonicals and missing previews.
• Developers: run it on a schedule or through the API and compare the issue counts after each release.

What it does

• Starts from your URLs and follows links on the same site (the start host, plus the host it redirects to, such as www.), breadth-first, up to the depth and page limits you set. Links to other sites are counted, never crawled.
• For each HTML page it records: HTTP status and redirects, title and length, meta description and length, the first H1 and the number of H1-H6 headings, skipped heading levels, canonical (self, other or missing), meta robots and X-Robots-Tag (indexable or not), lang, viewport, images without an alt attribute, word count, hreflang links, Open Graph and Twitter card tags, schema.org types (JSON-LD and microdata) and invalid JSON-LD blocks, and the number of internal and external links.
• Each page gets a list of issues (error, warning or notice) with a short explanation, for example title_missing, h1_multiple, canonical_points_elsewhere, noindex, images_missing_alt.
• The SUMMARY record adds totals per issue and the duplicate titles and meta descriptions across the pages.
• Pages answering HTTP 4xx/5xx are listed as free error rows, so you also see broken internal pages.

Polite by design

• Identifies itself as KelpForestLabs-SEOAudit with a link to this page; it never pretends to be a browser.
• Honours robots.txt (its own token KelpForestLabs-SEOAudit, else *) and Crawl-delay up to 10 seconds. robots.txt is checked for every redirect hop, on the host the redirect points to; a redirect into a disallowed path is not followed and not charged. If robots.txt answers 5xx or 429 or cannot be reached (network error, timeout), the whole host is treated as disallowed (RFC 9309). "Same site" means the same host name and port.
• Waits between requests to the same host (0.5 s by default) and does not follow links with query strings unless you ask.
• Never logs in, never sends cookies, and only reads public pages. Private and internal network addresses are refused.

Pricing (pay per event)

Event	Price
page (one HTML page fetched and audited)	US$0.006

Failed pages are free: pages answering HTTP 400 or higher, timeouts, DNS and network errors, pages robots.txt disallows, private addresses, non-HTML files and pages that redirect to another site give a free row.

Each run also has Apify's run-start charge of US$0.00005, once per run for up to 1 GB of memory plus once per extra GB (US$0.00005 at the default 512 MB), charged when the run starts, even if every page fails. maxPages caps the number of page requests, so 50 pages cost at most US$0.30 plus the start charge. You can also set a maximum charge per run in the Apify Console.

How to use

1. Add one or more start URLs (for example your home page).
2. Set Max pages to request and Crawl depth (0 = the start pages only).
3. Run the actor. Open the Pages view for one row per page, or the Issues per page view for one row per issue.
4. Open the SUMMARY record in the key-value store for totals and duplicate titles.

Input example

{"startUrls": [{"url": "https://docs.apify.com/"}], "maxPages": 10, "maxDepth": 1}

Output example (local fixture run, Oct 12, 2026; trimmed)

{"url": "https://example.com/guide", "status": "ok", "event": "page", "httpStatus": 200,
 "title": "Kelp care guide: grow healthy kelp at home", "titleLength": 42, "metaDescription": null,
 "h1": "Kelp care", "h1Count": 1, "headingSkips": ["h1->h3"], "canonicalStatus": "self", "indexable": true,
 "lang": "en", "viewport": true, "imagesCount": 1, "imagesMissingAlt": 1, "schemaTypes": ["Article"],
 "issues": [{"code": "meta_description_missing", "severity": "warning",
             "message": "No meta description; search engines will pick a snippet themselves."},
            {"code": "images_missing_alt", "severity": "warning", "message": "1 of 1 images have no alt attribute."}],
 "errorsCount": 0, "warningsCount": 2, "noticesCount": 3}

FAQ

Which thresholds does it use? Title 30-60 characters, meta description 70-160 characters, at least 200 words of visible text, one H1. These are common SEO-tool defaults, not rules from any search engine; treat warnings and notices as hints.

Does it run JavaScript? No. It reads the HTML the server sends. Pages that build their title or content in the browser may show as missing.

What does a failed page cost? Nothing. Only pages that were fetched (HTTP 200, HTML) and audited are billed.

Why was a page skipped? The row's error field says why: robots.txt, a private address, not HTML, or a redirect to another site.

Limits: what this actor does not do

• It does not render JavaScript, measure page speed or Core Web Vitals, or check backlinks and rankings.
• It does not check links to other sites (use a link checker for that) and does not crawl other hosts.
• It does not validate structured data against Google's rich-result rules; it lists the schema.org types it finds.
• It does not log in, solve captchas or use proxies; pages behind a login or bot protection show as error rows.
• It does not collect personal data: only page metadata (title, headings, tags) is kept.

Changelog

• 0.2 (October 15): robots.txt is checked on every redirect hop (also on another host a start URL redirects to); an unreachable or rate-limited (429) robots.txt now means "disallow everything"; another port is another site.
• 0.1 (October 2026): first version. Update (October 13): the home page linked as https://site.com and https://site.com/ is audited and billed once, and the SUMMARY totals are always filled.
• Update (October 14): with several start URLs, each one follows links to its own maxDepth, even through pages another start URL already audited (those are not requested, audited or billed again).