Schema Markup Validation Crawler

Actor #196 inspects the initial static HTML of explicit public web-page URLs and writes one isolated result per requested URL. Version 1 extracts JSON-LD blocks (<script type="application/ld+json">) with a schema.org signal and returns bounded local diagnostics plus safe fetch/input errors.

What it is for

SEO teams, content agencies, and AI-search analytics vendors can use the dataset to inventory JSON-LD that is present in a page response, identify obvious local structural issues, and feed deterministic records into their own reporting or alerting workflows.

This actor is not a Google Rich Results Test, a Schema.org validator service, an SEO eligibility/ranking scorer, or a content-policy checker. A completed record is not a claim that markup is valid, eligible, approved, truthful, or complete.

Input

startUrls is required and must contain at least one entry: an ordered array of objects, each with a string url.

{
  "startUrls": [
    { "url": "https://example.com/page" },
    { "url": "https://example.org/article" }
  ]
}

The request-list schema default, prefill, and sample_input.json use { "url": "https://example.com/" }, the same object shape required for every entry. This public page provides a safe, meaningful no_markup example record. URL strings are rejected. Input order and duplicate items are preserved; every entry gets its own terminal record.

Input and network limits

• Only absolute public http: and https: URLs are accepted.
• URLs with credentials, IP-literal hosts, non-default ports, or a hostname resolving to a non-public address are rejected before fetching.
• Redirect destinations are rechecked; at most five redirects are followed.
• The runtime fetches initial HTML only, accepts HTML/XHTML responses, caps response bodies at 2 MiB, and retries only transient transport failures plus HTTP 408, 429, and 5xx responses (at most three attempts total).
• URL parsing is delegated to the runtime URL implementation. It lowercases scheme/host, removes fragments/default ports, and can percent-encode or normalize paths; callers that require byte-for-byte request-path preservation should retain their original input value (inputUrl).

Output

The default dataset contains one terminal record per input item. Core fields are:

Field	Meaning
inputIndex, inputUrl	Original request position and supplied string.
normalizedRequestedUrl, finalUrl	Accepted URL and final response URL, or null when unavailable.
status	completed, completed_with_findings, no_markup, fetch_error, or input_error. This is processing state, not an SEO verdict.
markup	Extracted JSON-LD items with source locator, types, schema signals, bounded evidence, and local findings.
summary	Candidate, finding, and ignored-JSON-LD counts.
http, error, processing	Safe response metadata, stable error detail, and contract version/timestamp.

A JSON-LD script that cannot parse remains an item-level JSONLD_INVALID_JSON finding; it does not discard later JSON-LD blocks or later URLs. The v1 local rules additionally warn when a schema candidate has no usable @type or a declared non-@ property is null, blank, or an empty array. Findings are capped at 20 per markup item.

Deliberate v1 boundaries

Not implemented: Microdata, RDFa, browser rendering, dynamic/post-load markup, remote validator calls, a type-specific schema rule catalog, crawled links/sitemaps, authentication/cookies/custom headers, repair/generation, billing-event charging, Cloud deployment, or Store publication.

Raw HTML is never emitted. JSON-LD evidence is capped at 1,000 characters per item. This actor does not make claims about Google eligibility, ranking, policy compliance, factual accuracy, or complete schema.org conformance.

Local development

npm ci
npm test
npx --yes apify-cli@latest validate-schema < /dev/null

src/main.js uses the normal Apify input/dataset interfaces: the platform injects INPUT in the default Key-Value Store and records are pushed to the default dataset. The npm start entrypoint is therefore intended for an Apify platform run or a local run that initializes those standard storages. The deterministic unit suite exercises the extraction contract without relying on a live external page.

The Docker runtime uses apify/actor-node:22, installs production dependencies with npm ci --omit=dev, and starts src/main.js through npm start.

Store-positioning recommendation (not activated)

If a later review and Cloud smoke gate approve a Store release, position it as a static JSON-LD audit/inventory utility for technical SEO workflows—not as a universal Schema.org validator or Google-eligibility checker. The roadmap buyer fit is SEO teams, content agencies, and AI-search analytics vendors. Recommend an Apify Store category only after verifying the then-current category vocabulary; MARKETING is the closest currently known general category, but no public/category/pricing change is configured here.

The roadmap records a future primary result event named schema-markup-validation-crawler-result and launch guidance of $1.50 per 1,000 raw results. This source does not emit a billing event, and no current pricing configuration was verified or activated. Do not make a pay-per-result Store claim until a separately reviewed event adapter, current Apify pricing verification, an exact-default Cloud run, and a non-empty dataset/output check are complete.