Structured Data Extractor — JSON-LD, Open Graph & Meta Tags

Extract every machine-readable block from any web page in one pass: JSON-LD (schema.org) with all detected types, Open Graph, Twitter Card, standard meta tags, canonical URL, hreflang alternates and favicons — one clean JSON item per URL. Fetches only the URLs you provide, never crawls. No credentials or API keys needed.

What it does

Give it a list of page URLs. For each page you get one dataset item with:

• jsonLd[] — every application/ld+json block, parsed, in page order (broken blocks are counted in jsonLdErrors, never crash the run)
• jsonLdTypes[] — deduplicated schema.org @type values found anywhere in the JSON-LD, including inside @graph and nested entities (e.g. ["Article", "Person", "BreadcrumbList"])
• openGraph — og:* tags without the prefix; repeated keys (multiple og:image) become arrays
• twitterCard — twitter:* tags without the prefix
• meta — title, description, robots, author, generator, keywords, language, charset
• canonical — resolved to an absolute URL
• hreflang[] — language alternates as {lang, url} (absolute)
• icons[] — favicon / apple-touch-icon links with sizes
• stats — block and tag counts

Failed downloads (404s, non-HTML responses, oversized pages) produce an item with status: "failed" and a clear error, so batch jobs always finish.

Input

Field	Type	Default	Description
urls	array	—	Page URLs (only these are fetched; no crawling)
timeoutSecs	integer	20	Download timeout per page
maxFileSizeMb	integer	5	Larger pages are skipped with a failed item

Example output (abridged)

{
    "url": "https://example.com/article",
    "status": "success",
    "pageTitle": "Fruit Report — the Quarterly Edition",
    "meta": {"description": "A quarterly report about fruit.", "language": "en-GB"},
    "canonical": "https://example.com/reports/fruit-quarterly",
    "jsonLd": [{"@context": "https://schema.org", "@type": "Article", "headline": "Fruit Report"}],
    "jsonLdTypes": ["Article", "Person"],
    "jsonLdErrors": 0,
    "openGraph": {"title": "Fruit Report", "type": "article", "image": ["https://example.com/img/a.jpg", "https://example.com/img/b.jpg"]},
    "twitterCard": {"card": "summary_large_image", "title": "Fruit Report (Twitter)"},
    "hreflang": [{"lang": "ja", "url": "https://example.com/ja/fruit"}],
    "icons": [{"rel": "icon", "href": "https://example.com/favicon.ico", "sizes": "32x32"}],
    "stats": {"jsonLdBlocks": 1, "ogTags": 4, "twitterTags": 2}
}

Typical uses

• SEO audits at scale: which pages have schema markup, which types, where it's broken
• Verify social previews (Open Graph / Twitter Card) across a URL list before campaigns
• Harvest schema.org entities (Product, Article, Recipe, Event…) as clean JSON for pipelines
• Check canonical/hreflang consistency after site migrations

Limitations

• Static HTML only: markup injected by client-side JavaScript is not seen
• Microdata (itemscope/itemprop) and RDFa are not parsed — JSON-LD is the dominant modern format and the one search engines recommend
• Fetches the exact URLs given; pair it with a sitemap extractor for URL discovery

Development (local)

cd actors/structured-data-extractor
uv venv --python 3.13 .venv && uv pip install -p .venv/bin/python -r requirements.txt

.venv/bin/python tests/run_local_test.py           # end-to-end test (apify run equivalent), exit 0 = ALL PASS
../../node_modules/.bin/apify run                  # real apify CLI local run (input: storage/key_value_stores/default/INPUT.json)

Fixtures live in tests/fixtures/ (full/broken-jsonld/minimal/not-html); expected results in tests/expected_output.json. Publishing → ../../docs/publishing.md.