Web Metadata Extractor

Extract machine-useful metadata from web pages as structured JSON: page title, meta description, Open Graph tags, Twitter Card tags, other meta tags, JSON-LD structured data, canonical URL, declared language, and up to 10 H1 headings. Plain HTTP fetch + parsing — no AI, no browser, no JavaScript rendering.

Input

Field	Type	Required	Description
pageUrls	string[]	yes	Web page URLs to extract metadata from (min 1).
useResidentialProxy	boolean	no	Route fetches through residential proxies for bot-walled targets. Default false.

Output

One dataset item per input URL. Successful items carry url, finalUrl (after redirects), status: "ok", and extracted fields: title, description, language (from <html lang> or og:locale), canonicalUrl, openGraph, twitterCard, meta (other named meta tags), jsonLd, jsonLdTypes, and h1 (list of up to 10 H1 heading texts; H2–H6 are not extracted). Failed URLs produce an explicit error record (url, status: "error", error message) — one bad URL never kills the batch.

The key-value store record run-summary holds requested, succeeded, failed, billable_units, ppe_event, and useResidentialProxy.

If pageUrls is empty the Actor falls back to https://www.example.com and charges for it normally, so always pass the URLs you want.

Pricing

Pay Per Event: one page-processed event per page that is fetched and parsed successfully, at $0.001/page. The charge fires after the status: "ok" record is pushed to the dataset, even if the page has few or no metadata tags. Failed pages (HTTP error, network error, invalid URL) are not billed. Apify platform usage, including residential proxy when you enable it, is included in the event price and is not billed to you separately.

Limitations

• Plain HTTP fetch only: pages that require JavaScript rendering return whatever the raw HTML contains.
• No paywall bypass, no login, no CAPTCHA solving.
• Only the first 2 MB of each page's HTML is read; transient HTTP errors (429/5xx) and network errors are retried up to 3 times.
• Bot-walled sites may need useResidentialProxy: true (included in the event price).
• Pricing is provisional pending an actor-specific measured P95 cost benchmark.

Example (Apify API)

curl -X POST "https://api.apify.com/v2/acts/skilled_glee~web-metadata-extractor/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pageUrls": ["https://example.com", "https://www.iana.org"]}'

Version

0.3 — current Store build line (0.3.x). Output fields are exactly those listed above.