SEO Site Audit - crawl your site, get a ranked fix list

Technical SEO audit for a website you own, manage or have been asked to review. Give it a domain: it reads robots.txt, starts from the sitemap (or the page you name), follows same-site links up to your page limit, and returns one row per page with a 0-100 score and its issues, plus a site summary row with the issues ranked by severity. Plain HTTP, no browser, no proxy.

What it checks, per page

Area	Checks
Status	HTTP status, redirect chain (every hop), redirect loops, unreachable pages
Titles and meta	Title and meta description: missing, too short, too long, duplicated across pages
Headings	H1 missing or more than one
Indexing	Canonical (missing or pointing elsewhere), noindex (meta tag or X-Robots-Tag), hreflang
Content	Word count (thin pages), images without alt text, structured-data types (JSON-LD, microdata)
Links	Internal and external link counts, broken internal links (targets that answered 4xx/5xx in the same crawl)
Performance	Response time (ms), HTML page weight (bytes)
Mobile	Viewport meta tag
Security	Mixed content: http:// images, scripts, stylesheets and iframes on https pages (mixed_content, with mixedContentCount per page)

Severity scale: critical (a page that errors or cannot be reached), high (missing title or H1, broken internal links), medium (redirect chains, noindex, duplicates, missing descriptions, missing alt text, slow or heavy pages), low (length and housekeeping issues). The page score is 100 minus 40 per critical, 15 per high, 7 per medium and 2 per low issue.

How it behaves on your site

• User agent: KeencrestBot/1.0 (+https://keencrest-guides.web.app/). It is always honest about who it is.
• It follows robots.txt for every URL it requests, including a Crawl-delay, and reports how many URLs it skipped (robotsBlocked on the summary row). It cannot be switched off.
• Default pace is one request about every 0.5 seconds, 3 at a time. On HTTP 429 or 503 it waits (honouring Retry-After) and backs off. If the site keeps throttling, or serves a bot-challenge page (Cloudflare "Just a moment", CAPTCHA), it stops and says so in the summary. It never rotates IPs or works around a block.
• Only the starting host is crawled (www. and the bare domain are one crawl scope: a page is audited once whichever form links to it, and links to the other form are not reported as outside links). When one form redirects to the other, pages whose internal links still use the redirecting form get the links_to_redirecting_host issue (medium): point links and the sitemap at the final host. If your start URL itself redirects to another origin, the summary row says where in startUrlRedirectsTo. Links with query strings are not followed; URLs listed in the sitemap are audited as given.

Optional link check (off by default)

Set checkLinks to true and, after the crawl, every link on your pages that points to another site, or to your own URLs the crawl did not fetch (PDFs, images, pages past the page limit), is requested once and reported as ok, broken or unverified:

• It sends HEAD first and falls back to GET (without downloading the body) when HEAD fails or answers an error, so servers that refuse HEAD are not flagged by mistake.
• Each target host's robots.txt is read first. A URL it disallows is not requested, is reported unverified with robotsBlocked: true, and is free. Same honest user agent, same polite per-host pacing and Crawl-delay.
• broken means a 404/410/5xx answer, a name that does not resolve or a refused connection. unverified means the server refused a bot (401/403/429 and similar), timed out, or served a challenge page (never worked around). Social networks that block every bot are left out.
• Most-linked targets are checked first, up to maxLinkChecksPerSite (default 200, max 2,000). Broken outside links also appear on the page rows (brokenExternalLinks) and as the broken_external_links issue, and broken uncrawled internal targets join broken_internal_links.
• Every unverified row says why in unverifiedReason: robots (disallowed, not requested), timeout, 403, 429, challenge (bot-challenge page, never worked around) or other. The summary row counts them in linksUnverifiedByReason, and the run status reads like "3 unverified: 2 403, 1 timeout".
• Output: one linkCheck row per target (url, linkKind, status, linkResult, unverifiedReason, method, foundOn). Open the Link checks view.

Fix list

With includeFixList on (default), the dataset also gets one free row per issue: url, issue, severity, fixHint (what to change), worst first. Open the Fix list view, export the dataset as CSV with that view, or download FIX_LIST.csv from the run's key-value store.

Input

{
  "websites": ["https://example.com"],
  "maxPagesPerSite": 100,
  "checkLinks": false,
  "maxLinkChecksPerSite": 200,
  "includeFixList": true,
  "requestDelaySeconds": 0.5
}

Leave the input empty and it audits keencrest-guides.web.app (our own guides site, up to 5 pages) as a demo.

Output (one page row)

{
  "rowType": "page",
  "url": "https://example.com/pricing",
  "finalUrl": "https://example.com/pricing/",
  "status": 200,
  "score": 78,
  "worstSeverity": "medium",
  "issues": [
    {"code": "redirect_chain", "severity": "medium", "message": "Redirect chain of 2 or more hops"},
    {"code": "images_missing_alt", "severity": "medium", "message": "Images without alt text"}
  ],
  "title": "Pricing | Example",
  "titleLength": 17,
  "metaDescription": "Plans and prices for Example.",
  "h1Count": 1,
  "canonical": "https://example.com/pricing/",
  "noindex": false,
  "imagesTotal": 8,
  "imagesMissingAlt": 3,
  "wordCount": 612,
  "internalLinks": 41,
  "externalLinks": 3,
  "brokenInternalLinks": [],
  "redirectCount": 2,
  "responseTimeMs": 240,
  "pageWeightBytes": 58214,
  "structuredData": ["Organization", "Product"],
  "error": null,
  "failureReason": null,
  "scrapedAt": "2026-11-01T10:00:00+00:00"
}

failureReason is null when the page was read as HTML, otherwise one of dns_error, timeout, connection_error, blocked_403, rate_limited_429, http_<code>, not_html, robots_disallowed, invalid_url, crawl_error, so you can count failed pages by cause without parsing error.

The site summary row (rowType: "siteSummary", free) holds pagesAudited, averageScore, counts of pages per severity, and issues: the ranked table with code, severity, pagesAffected and up to 3 example URLs for each issue. Export the dataset as CSV or Excel for a fix list.

Pricing

$8.00 per 1,000 pages audited ($0.008 a page), pay per event. The summary row and the fix list rows are free. Pages that robots.txt disallows or that cannot be reached are not billed. Worked example: a 300-page site costs $2.40; a 5,000-page site $40.

Link check add-on: $1.00 per 1,000 links checked ($0.001 a link), only when checkLinks is on. Targets that robots.txt disallows are free. Worked example: a 100-page site with 200 outside links checked costs $0.80 for pages plus $0.20 for links, $1.00 in total. maxPagesPerSite, maxLinkChecksPerSite and your run's maximum charge all cap spend; the crawl never goes past what you can be billed for.

FAQ

Can I audit a competitor? Use it on sites you own, manage or have permission to audit. The crawl is polite and honours robots.txt, but the data is yours to use responsibly.

Does it run JavaScript? No. It audits the HTML the server returns, which is what search crawlers read first. Pages that only render content client-side will show thin content.

Why is a broken link not listed? Without checkLinks, a link is reported broken only when its target was itself audited in the same run and answered an error. Turn checkLinks on to test outside links and uncrawled targets too.

Why is a link "unverified"? The target refused a bot (many sites answer 403 or 429 to anything automated), timed out, or its robots.txt disallows the URL; the unverifiedReason column tells which. The Actor does not retry around it, so check those few by hand.

Is the data personal data? It is page metadata and link counts. No page text is stored beyond titles, descriptions and the first H1.