Technical SEO Site Audit

Crawl public pages or XML sitemaps and get deterministic, actionable technical SEO findings for every page, plus concise JSON and Markdown site summaries.

What it checks

• Titles, meta descriptions, headings, canonicals, robots directives, hreflang, images, and structured data
• Duplicate titles and descriptions across the audited site
• Internal links with cached HEAD checks and a GET fallback
• Ordinary, gzip-compressed, and nested sitemap indexes
• $0.004 per successfully fetched and saved page

This Actor is HTTP-only and does not render JavaScript.

Quick start

{"startUrls":[{"url":"https://example.com"}],"discoverSitemap":true,"maxPages":100,"checkLinks":true}

Use with AI agents / MCP

Call lintlab/seo-site-qa through the Apify API or Apify MCP server. Use the per-page findings for precise fixes and SUMMARY or SUMMARY.md for a site-level plan.

Overview

SEO Site QA is an HTTP-only technical SEO audit for sites you own or are authorized to audit. It accepts pages, an XML sitemap, or a sitemap index; checks each allowed page; and writes compact, deterministic JSON plus a site summary. It is intended for developers, agencies, CI workflows, and AI agents that need findings they can act on without running a browser.

What it does

• Reads ordinary and gzip-compressed sitemaps, including nested sitemap indexes.
• Optionally discovers a sitemap from robots.txt and then /sitemap.xml when given only a homepage.
• Uses Crawlee's CheerioCrawler with guarded HTTP requests. It does not launch a browser.
• Audits up to 5,000 pages and can optionally discover more pages from internal links.
• Checks internal links with cached HEAD requests and a GET fallback, within a configurable cap.
• Produces one dataset item per attempted page, SUMMARY JSON, and SUMMARY.md.
• Charges only after an allowed, successfully fetched page has been saved to the dataset.

Input

{
  "startUrls": [{ "url": "https://crawlee.dev" }],
  "discoverSitemap": true,
  "maxPages": 100,
  "sameHostOnly": true,
  "followLinks": false,
  "checkLinks": true,
  "maxLinkChecks": 500,
  "timeoutSecs": 20,
  "concurrency": 5
}

You may use sitemapUrl instead of, or together with, startUrls. startUrls also accepts plain URL strings through the programmatic input path. Global concurrency defaults to 5; requests to one origin are limited to 2 concurrently.

Page output

Each dataset item describes one attempted page. Blocked and failed pages include an error and are not charged.

{
  "url": "https://example.com/about",
  "finalUrl": "https://example.com/about",
  "status": 200,
  "redirectChain": [],
  "title": "About Example Company",
  "metaDescription": "How our team builds useful products.",
  "h1": ["About us"],
  "canonical": "https://example.com/about",
  "indexable": true,
  "wordCount": 438,
  "jsonLdTypes": ["AboutPage"],
  "findings": [
    {
      "checkId": "opengraph.image_missing",
      "severity": "notice",
      "message": "Open Graph image is missing.",
      "fix": "Add a og:image meta tag for richer social previews."
    }
  ],
  "counts": { "error": 0, "warning": 0, "notice": 1 },
  "checkedAt": "2026-01-01T00:00:00.000Z"
}

redirectChain contains {url, status, location} objects. A finding always contains checkId, severity, message, and fix; value is included when useful.

Checks

Check IDs are stable API values. New checks may be added in later versions.

Check ID	Severity	Condition
title.missing	error	No <title> element
title.empty	error	Title element has no text
title.too_long	warning	Title is over 60 characters
title.too_short	warning	Title is under 10 characters
title.duplicate	warning	Same non-empty title appears on multiple audited pages
description.missing	warning	Meta description is absent or empty
description.too_long	warning	Meta description is over 160 characters
description.duplicate	warning	Same non-empty description appears on multiple audited pages
h1.missing	error	No H1 heading
h1.multiple	warning	More than one H1
headings.level_skip	warning	Heading order jumps by more than one level
canonical.missing	warning	Canonical link is absent or empty
canonical.relative	warning	Canonical is not written as an absolute URL
canonical.elsewhere	warning	Canonical resolves to a different URL
canonical.malformed	error	Canonical cannot be parsed as a URL
canonical.non_200	error	Canonical target does not return HTTP 200 or cannot be fetched
robots.meta_noindex	warning	Meta robots contains noindex
robots.meta_nofollow	warning	Meta robots contains nofollow
robots.x_noindex	warning	X-Robots-Tag contains noindex
robots.x_nofollow	warning	X-Robots-Tag contains nofollow
robots.blocked_sitemap	error	URL is in a sitemap but disallowed by robots.txt
robots.disallowed	error	Non-sitemap start/discovered URL is disallowed
http.non_200	error	Final page response is outside 2xx
sitemap.redirected	warning	A URL taken from the sitemap redirects
html.lang_missing	warning	<html> has no lang
html.lang_malformed	warning	lang is not a valid BCP 47 tag
viewport.missing	warning	Viewport meta tag is absent
images.alt_missing	warning	One or more images lack an alt attribute; value has count and first five sources
opengraph.title_missing	notice	og:title is absent or empty
opengraph.description_missing	notice	og:description is absent or empty
opengraph.image_missing	notice	og:image is absent or empty
jsonld.invalid	error	A JSON-LD block does not parse as JSON
jsonld.types	notice	Valid JSON-LD types were found; types are reported in value and jsonLdTypes
resources.mixed_content	error	An HTTPS page references an http:// resource
html.too_large	warning	Uncompressed HTML is over 1.5 MB
hreflang.malformed	warning	Hreflang language or absolute HTTP(S) target is malformed
links.broken_internal	error	Internal link returns 4xx/5xx or a network error
links.redirect_chain	warning	Internal link has more than one redirect hop
fetch.failed	error	Page fetch fails before an HTTP response is available

An empty JSON-LD set is not an error. If valid JSON-LD exists, its @type values are reported. Link findings are attached to each source page; the summary also lists broken source-to-target relationships.

Summary and score

The default key-value store receives:

• SUMMARY: JSON with pages audited/attempted, aggregate counts, top issues, duplicate title and description groups, broken links, sitemap errors, and the score.
• SUMMARY.md: the same core results in a readable Markdown report.

The score is:

100 - min(100, (errors × 5 + warnings × 2 + notices × 0.25) / successful pages)

It is clamped to 0–100 and rounded to one decimal. The formula is deliberately simple and comparative; a business-critical issue can matter more than its numeric weight.

Pricing

The page-audited event costs $0.004 per successfully audited page, or $4 per 1,000 pages. Internal link checks are free and bounded by maxLinkChecks. Disallowed pages, network failures, and non-2xx final page responses are written for diagnosis but are not charged. The Actor stops writing further page results when the platform reports that the event charge limit has been reached.

Limits and behavior

• Maximum 5,000 scheduled pages per run and 10,000 unique internal link checks.
• Maximum 10 redirects per HTTP operation.
• HTML responses are capped at 10 MB; sitemap responses at 20 MB.
• Sitemap indexes are capped at 100 fetched sitemap documents per run.
• Link checks use HEAD, falling back to GET for HTTP 405 or 501. Canonical targets are checked separately and cached.
• sameHostOnly compares hostnames. URL fragments are removed during normalization.
• The crawler evaluates server-delivered HTML only. It does not execute JavaScript, measure rendered layout, or run Core Web Vitals.
• Hreflang validation checks language syntax and absolute HTTP(S) targets; it does not verify reciprocal annotations.
• Duplicate matching is case-insensitive after trimming, not semantic.
• robots.txt rules support user-agent groups, Allow/Disallow, *, and terminal $; unusual nonstandard directives are ignored.

Safety and compliance

Use this Actor only for sites you own or have permission to audit.

• User agent: lintlab-seo-qa/1.0 (+https://github.com/lintlab); robots product token: lintlab-seo-qa.
• robots.txt is cached per origin. Fetch/network errors and 5xx responses fail closed (all URLs on that origin are disallowed); 4xx responses allow crawling under RFC 9309 behavior.
• Every page, sitemap, canonical, link check, and redirect hop is restricted to HTTP(S), DNS-resolved before connecting, and blocked if any result is private, loopback, link-local, CGNAT, multicast, reserved, unspecified, or IPv6 ULA/IPv4-mapped private space.
• Redirects are followed manually, at most 10 hops, so every destination receives the same SSRF validation.
• No proxy, login/cookie input, CAPTCHA handling, browser, or stealth behavior is used.
• The output does not extract emails, phone numbers, or personal names.

Local development

Requires Node.js 20 or newer.

npm ci
npm test
APIFY_LOCAL_STORAGE_DIR=./storage CRAWLEE_STORAGE_DIR=./storage node src/main.js

Before the final command, write the desired Actor input to storage/key_value_stores/default/INPUT.json (or use apify run --purge with the Apify CLI).

---

Built by lintlab — small, reliable data tools. AI-assisted (Claude/Codex), reviewed before release. Support: lintlab.dev@gmail.com · https://github.com/lintlab