Page Metadata, OpenGraph & JSON-LD Extractor

Bulk meta tag extractor for SEO, social previews and structured data. Paste a list of public URLs and get one clean JSON row per page: title, meta description, canonical, robots, language, OpenGraph and Twitter card tags, parsed schema.org JSON-LD and microdata, hreflang, RSS/Atom feeds, favicons, headings, link counts and the detected tech stack (CMS, framework, analytics, CDN, server) with the evidence for every match. $0.60 per 1,000 pages. Failed pages are free.

What does Page Metadata Extractor do?

• Fetches each URL with a fast HTTP client (no browser), following redirects safely, and decodes any character set (UTF-8, Shift_JIS, windows-1252 and more).
• Extracts all standard <head> metadata and social-preview tags (og:*, twitter:*, article:*, product:*).
• Parses every JSON-LD block and lists all schema.org types (Product, Article, FAQPage, LocalBusiness, BreadcrumbList...). Invalid JSON-LD is counted and flagged.
• Detects 55 common technologies from headers, meta tags, script URLs and HTML, and shows why each was detected.
• Flags common SEO and social issues per page: MISSING_TITLE, MISSING_META_DESCRIPTION, MISSING_OG_TITLE, MISSING_OG_IMAGE, MISSING_TWITTER_CARD, INVALID_JSON_LD, NO_STRUCTURED_DATA, MISSING_CANONICAL, MISSING_HTML_LANG.
• Three ready-made table views: Metadata, Social previews (with og:image thumbnails) and Tech stack.

Why use it?

• Cheap at scale: about 8x cheaper than typical metadata scrapers, and you only pay for pages that were actually extracted.
• Clean, typed JSON that is easy to load into Google Sheets, BigQuery, a CRM or an LLM pipeline.
• Privacy by design: personal data is removed before it reaches your dataset (see below).
• Polite and safe: honest user agent, robots.txt and Crawl-delay respected, at most 2 parallel requests per site, automatic retries with backoff for timeouts, 429 and 5xx.

How to use it

1. Click Try for free (or Start), paste your URLs into URLs (one per line, or upload a text/CSV file).
2. Keep the defaults or switch off the parts you don't need.
3. Click Start. Download the results as JSON, CSV, Excel or HTML, or read them through the API.

Tip: to audit a whole website, first list its pages with our Sitemap URL Extractor & SEO Health Checker, then feed the URLs here.

Input

Field	Default	What it does
urls	required	Up to 100,000 public URLs per run ([{ "url": "https://..." }], plain strings also work).
includeOpenGraph	true	OpenGraph, Twitter cards and other og namespaces.
includeJsonLd	true	Parsed JSON-LD, schema.org types, microdata types.
detectTechnologies	true	Tech stack with evidence.
countLinks	true	Internal / external / nofollow link counts.
respectRobotsTxt	true	Skip (free) URLs that robots.txt disallows; honour Crawl-delay up to 10 s.
maxConcurrency	10	Parallel requests overall (max 2 per site).
requestTimeoutSecs	20	Timeout per request.
maxRetries	1	Retries for timeouts, network errors, 408/429/5xx.
userAgent	PageMetadataBot/1.0	Honest bot user agent.

{ "urls": [{ "url": "https://www.python.org" }, { "url": "https://data.gov.my" }], "detectTechnologies": true }

Output

One row per URL. Example (shortened):

{
  "url": "https://www.python.org/",
  "finalUrl": "https://www.python.org/",
  "statusCode": 200,
  "responseMs": 143,
  "title": "Welcome to Python.org",
  "metaDescription": "The official home of the Python Programming Language",
  "canonical": null,
  "lang": "en",
  "openGraph": { "type": "website", "site_name": "Python.org", "title": "Welcome to Python.org", "image": "https://www.python.org/static/opengraph-icon-200x200.png" },
  "twitterCard": {},
  "jsonLdTypes": ["WebSite", "SearchAction"],
  "technologies": [
    { "name": "jQuery", "category": "JS library", "evidence": "script https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js" },
    { "name": "Fastly", "category": "CDN", "evidence": "header x-served-by: cache-iad-..." },
    { "name": "Nginx", "category": "Web server", "evidence": "header server: nginx" }
  ],
  "linkCounts": { "internal": 134, "external": 79, "nofollow": 0 },
  "personalDataRedactions": 0,
  "issues": ["MISSING_TWITTER_CARD", "MISSING_CANONICAL"],
  "encoding": "utf-8",
  "extractedAt": "2026-10-26T10:35:12.345Z"
}

Pages that could not be extracted are still listed, with error and a machine-readable errorCode (HTTP_404, HTTP_503, TIMEOUT, NETWORK_ERROR, NOT_HTML, ROBOTS_DISALLOWED, BLOCKED_NON_PUBLIC, INVALID_URL, TOO_MANY_REDIRECTS). These rows are not charged. A SUMMARY record in the key-value store has the run totals.

How much does it cost?

Pay per event. You pay only for pages that were fetched and extracted successfully.

Event	Price
Page extracted	$0.0006

• 1,000 pages ≈ $0.60; 10,000 pages ≈ $6.
• Apify adds its standard tiny run-start fee (about $0.00005 per GB of memory; the default run uses 512 MB).
• Errors, 4xx/5xx, non-HTML files, blocked and robots.txt-disallowed URLs are free.
• Set Maximum cost per run and the Actor stops cleanly before it would exceed it; everything extracted so far is kept.

Use it from code or an AI agent

Every Apify Actor is an API. Call it with the Apify API, the JavaScript or Python client, or as a tool in Apify's MCP server for Claude, ChatGPT, Cursor and other AI agents. Input is a plain JSON object with a urls array; output is one JSON object per URL with stable field names and errorCode values, so agents can branch on results without parsing text.

curl -X POST "https://api.apify.com/v2/acts/thriving_adder~page-metadata-extractor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" -H 'Content-Type: application/json' -d '{"urls":[{"url":"https://example.com"}]}'

What data does it collect?

Only page-level technical metadata that websites publish for search engines and social networks. Personal data is removed by design:

• Never collected: author, article:author, book:author, profile:*, twitter:creator, fb:admins, og:email, og:phone_number.
• schema.org Person, Patient and ProfilePage nodes are reduced to a stub, and personal fields (email, telephone, birthDate, address, givenName, familyName...) are stripped everywhere. Business contact details of an Organization or LocalBusiness are kept.
• Each row reports how many fields were redacted in personalDataRedactions.

The Actor does not log in or use cookies, rejects localhost and private-network addresses (including redirects and DNS tricks), and respects robots.txt by default. Only use it on pages you are allowed to access.

Use cases

• SEO audits: find missing titles, descriptions, canonicals and hreflang across thousands of URLs.
• Social preview QA: check og:image and Twitter cards before a campaign.
• Structured data checks: confirm Product, Article, FAQ or LocalBusiness markup at scale.
• Technology and market research: which CMS, framework, analytics or CDN a list of public websites uses.
• AI and RAG pipelines: clean page context (title, description, types, language) for agents.

FAQ

Does it render JavaScript? No. It reads the HTML the server sends, which is what most search and social crawlers see. Pages that inject metadata only in the browser will show less.

Why was a page skipped? Check its errorCode. ROBOTS_DISALLOWED means the site asks bots not to crawl that page; you can switch off Respect robots.txt for your own sites.

How accurate is tech detection? Each match includes its evidence, so you can verify it. The rules are conservative to avoid false positives.

Problems or ideas? Open an issue in the Issues tab with the URL and input. We answer within 2 business days.