Site Launch & Migration Technical SEO Audit

Run a bulk technical launch and indexability audit across a flat list of URLs, a sitemap, or a chained dataset — before you flip a redesign live or after a migration. This Actor fetches each page over plain HTTP, follows redirects, reads the server-rendered HTML and response headers, and returns a deterministic PASS / WARN / FAIL verdict per URL plus a batch-level deploymentGate your SEO, engineering and QA teams can act on.

The most common launch catastrophe is an accidental noindex or a robots.txt Disallow that silently de-indexes a whole section. This audit treats that as the number-one signal.

What it checks per URL

• HTTP status and full redirect chain — every hop, loops, missing Location, long chains, HTTPS→HTTP downgrades and the final status code.
• Indexability (the core) — robots.txt allow/disallow for the URL path (fetched and parsed per host, with Google's most-specific-rule semantics), meta robots noindex, the X-Robots-Tag response header, and a self / cross-domain canonical check. noindex is detected from all three sources and reported in noindexSources.
• Core on-page tags — title (presence + length), meta description (presence + length), H1 count, canonical, and Open Graph (og:title, og:image). These are checked only on indexable HTTP 200 HTML pages, never on redirects or noindex pages, so you never get a false FAIL for a missing title on a page that was never meant to render one.
• JSON-LD — structural/syntactic validation only: is the ld+json valid JSON and does it declare an @type? (No schema.org scoring, no "AI-readiness".)
• hreflang — presence and sanity: valid language codes, a self-referencing entry and no conflicting language→URL mappings.

Site-level checks (once per run)

siteChecks reports the primary host, whether robots.txt was found, any sitemap it declares, whether a provided sitemap was parsable and how many URLs it held, and how much of the audited sample was indexable.

How verdicts work

• FAIL (blocks the deployment gate) — unreachable / DNS / timeout / redirect loop, a broken final status (4xx/5xx), an HTTPS downgrade, or indexability blockers: noindex (meta or header) and robots.txt disallow.
• WARN — quality risks that rarely block a launch: long redirect chains, missing/duplicate on-page tags, canonical pointing elsewhere, invalid JSON-LD, or hreflang sanity issues.
• PASS — indexable, reachable and free of the above.

Set treatWarnAsFail: true to make the batch gate stricter (any WARN fails the gate) without changing the per-URL diagnostic verdicts. deploymentGate is FAIL when any URL fails.

Input

Provide URLs in any combination (they merge and deduplicate):

{
  "urls": ["https://www.example.com/", "https://www.example.com/pricing"],
  "urlsText": "https://www.example.com/blog\n# comments are ignored\nhttps://www.example.com/help",
  "sitemapUrl": "https://www.example.com/sitemap.xml",
  "datasetId": "<another Actor run's datasetId>",
  "datasetUrlField": "url",
  "checkRobotsTxt": true,
  "checkOnPage": true,
  "checkStructuredData": true,
  "checkHreflang": true,
  "treatWarnAsFail": false
}

A sitemap is expanded (a bounded sampleSitemapUrls sample; a sitemap index is followed one level deep). datasetId reads URLs from another Actor for chaining.

This is not a source→target redirect-map validator. It audits each URL on its own merits. If you need to validate an old-URL → new-URL migration spreadsheet (301/308 correctness, exact destination match, many-to-one consolidation), use the companion Actor site-migration-redirect-map-qa — see cross-sell below.

Output

• Dataset — one PASS/WARN/FAIL record per URL (JSON/CSV/Excel), same shape family as the redirect-map QA Actor for easy side-by-side reporting.
• Key-value store — OUTPUT (JSON summary + siteChecks), REPORT (self-contained, CSP-protected HTML report) and SUMMARY (Markdown for tickets and pull requests).

Common issue codes: NOINDEX_META, NOINDEX_HEADER, ROBOTS_TXT_DISALLOWED, NOT_FOUND, BROKEN_STATUS, HTTPS_DOWNGRADE, REDIRECT_CHAIN_LONG, CANONICAL_MISSING, CANONICAL_CROSS_DOMAIN, TITLE_MISSING, TITLE_TOO_LONG, META_DESCRIPTION_MISSING, H1_MISSING, H1_MULTIPLE, OG_TITLE_MISSING, OG_IMAGE_MISSING, JSONLD_INVALID, JSONLD_NO_TYPE, HREFLANG_INVALID_CODE, HREFLANG_NO_SELF, HREFLANG_CONFLICT.

Where this fits: one migration gate, three stages

1. Redirect-map QA — validate the old→new URL spreadsheet (301/308, exact destination, chains, loops): Actor site-migration-redirect-map-qa.
2. Launch & indexability audit (this Actor) — is each live URL reachable, crawlable, indexable and technically sound?
3. AI-search / answerability readiness — is the content structured for LLM answer engines? That is a deliberately separate scope this Actor does not cover.

Important limitation — no JavaScript rendering

This Actor uses deterministic HTTP and server-rendered HTML only. It does not execute JavaScript, log in, or bypass bot protection. Tags injected client-side (canonical, robots, JSON-LD, hreflang) are not seen; such pages are marked confidence: LOW and should be verified in a rendered browser. These limits keep audits reproducible, inexpensive and low-maintenance.

Security and responsible use

• Audit only public URLs you own or are authorized to test.
• The Actor blocks localhost, private/link-local/reserved IP ranges, cloud metadata hosts, URL credentials and non-standard ports — and re-validates every redirect hop, including for robots.txt and sitemap fetches, closing DNS-rebinding gaps.
• Responses are capped, requests time out and concurrency is bounded.
• No personal data, login, proxy or external API is required.
• This is technical QA, not a guarantee of rankings or legal/SEO advice.

Pricing (pay per event)

• $0.007 per audited URL that returns a resolved final HTTP response (that is $7.00 per 1,000 URLs), plus a near-zero $0.00005 Actor start event.
• A URL is charged whenever it resolves to a final HTTP response, regardless of PASS/WARN/FAIL — a detected 404 or a noindex is a result you are paying for.
• Blocked, invalid, DNS-failed, timed-out and unreachable URLs are never charged, even though they are still delivered to the dataset.
• Platform usage is covered by the Actor creator, so the event price is what you pay.
• Set maxTotalChargeUsd on the run to cap spend. The Actor trims the input before doing any work, so your limit is never exceeded and skippedByChargeLimit reports what was left out.

Sources

• Google: Site moves with URL changes
• Google: Robots meta tag and X-Robots-Tag
• Google: robots.txt specification
• Google: Canonical URLs
• sitemaps.org protocol