Website Metadata & Open Graph Scraper

Extract metadata from public HTTP and HTTPS pages. Get the page title, meta description, canonical URL, robots meta tag, Open Graph fields, Twitter Card fields, and named meta tags.

Add 1–100 URLs. The Actor makes direct HTTP requests, returns one Dataset row for each input item, keeps repeated metadata values, and resolves relative metadata URLs. It does not crawl links or run JavaScript.

Table of contents

• What it does
• What data you get
• Quick start
• Input
• Input example
• Output example
• Output fields
• Pricing and limits
• Use cases
• Scheduling and monitoring
• API, export, and integrations
• Limits and partial results
• Troubleshooting
• FAQ
• Support

What it does

The Actor fetches only the URLs you provide and reads metadata from the returned HTML. It follows bounded HTTP redirects, but it does not discover or crawl other pages.

Exact duplicate input URLs share one network fetch, while each input item still gets its own output row. If the same meta key appears more than once, the output keeps all values in source order.

What data you get

Each processed input can include:

• HTML title and meta description.
• Canonical URL and robots meta value.
• All og:* Open Graph values.
• All twitter:* Twitter Card values.
• Named/property meta tags in metaTags, with URL-like values resolved.
• HTTP status, final URL, content type, timing, attempts, and result status.
• Safe error fields for invalid, blocked, missing, or failed URLs.

Relative canonical, Open Graph URL/image, and Twitter URL/image values are resolved against the final page URL. Missing metadata stays null or an empty object.

Quick start

1. Open the Input tab.
2. Add one public URL to urls, for example https://example.com/.
3. Keep the default request settings for a normal run.
4. Set maxTotalChargeUsd high enough for the possible event charges, then click Start.
5. Open Dataset and check resultStatus, title, and the metadata fields.

For one possible useful result, allow at least $0.0028 in event charges. For 100 possible useful results, allow at least $0.181. Platform usage is billed separately by Apify for this Actor.

Input

Field	Plain name	Type	Required	Runtime default	UI prefill	Allowed values	Effect	Recommendation
urls	URLs	array of strings	Yes	None	https://example.com/, https://apify.com/	1–100 HTTP(S) URLs, max 2048 characters each	Pages to fetch. Exact duplicates share one fetch but keep one output row per input item.	Start with a small URL list, then increase only when the source pages respond normally.
maxConcurrency	Concurrent requests	integer	No	10	None	1–20	Maximum URLs fetched at the same time.	Keep 10 unless the target sites are slow or rate-limited.
requestTimeoutSecs	Request timeout	integer	No	20	None	3–60	Maximum time for one HTTP attempt.	Increase only for known slow pages.
maxRetries	Retries	integer	No	1	None	0–3	Retries temporary network errors, HTTP 429, and HTTP 5xx.	Keep 1 for normal runs.

Empty urls, more than 100 items, unknown input fields, and wrong types are rejected before fetch work starts. API input {} is invalid; the UI prefill is a small demo, not a runtime URL default.

maxRetries: 1 allows one retry after the first failed attempt. A URL can make more requests when it redirects (up to 5 redirects). The timeout limits each HTTP attempt; retries and redirects can increase total time. Lower concurrency for rate-limited sites. The platform run timeout can stop a long list before every item is processed.

Input example

{
  "urls": [
    "https://example.com/",
    "https://www.w3.org/"
  ],
  "maxConcurrency": 2,
  "requestTimeoutSecs": 20,
  "maxRetries": 1
}

This run fetches the two supplied pages and returns two Dataset rows when both items are processed.

Output example

This is one real row from https://example.com/, trimmed only to remove timing fields:

{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "ok": true,
  "status": 200,
  "title": "Example Domain",
  "description": null,
  "canonical": null,
  "robots": null,
  "openGraph": {},
  "twitter": {},
  "metaTags": {
    "viewport": "width=device-width,initial-scale=1"
  },
  "resultStatus": "complete",
  "errorType": null,
  "errorMessage": null
}

Output fields

Field	Type	Can be empty?	Example	Meaning
url	string	No	https://example.com/	Input URL; credential userinfo is redacted in rejected URLs.
finalUrl	string or null	Yes	https://example.com/	URL after redirects, when a connection was made.
ok	boolean	No	true	true for complete or partial HTML results.
status	integer or null	Yes	200	HTTP status code when available.
contentType	string or null	Yes	text/html	Response Content-Type when available.
responseTimeMs	integer or null	Yes	120	Total URL fetch time, including redirects and retries.
attempts	integer	No	1	Total HTTP attempts for this URL.
fetchedAt	ISO string	No	2026-10-05T12:00:00Z	Time the row was created.
title	string or null	Yes	Example Domain	First HTML <title>.
description	string or null	Yes	null	First meta description; repeats remain in metaTags.description.
canonical	string or null	Yes	https://site.com/page	First canonical link, resolved to an absolute URL.
robots	string or null	Yes	index,follow	First robots meta value. Repeats remain in metaTags.robots.
openGraph	object	Yes	{"image":["https://site.com/a.png","https://site.com/b.png"]}	All og:* values. Repeated keys become arrays; URL-like values are resolved.
twitter	object	Yes	{"card":"summary"}	All twitter:* values. Repeated keys become arrays; URL-like values are resolved.
metaTags	object	Yes	{"description":["First","Second"]}	Named/property meta tags. Repeated keys become arrays.
resultStatus	string	No	complete	complete, partial, not_found, or error.
errorType	string or null	Yes	not_found	Stable category such as invalid_url, blocked_address, not_found, or content_too_large.
errorMessage	string or null	Yes	The page was not found.	Safe error detail for this item.

The default key-value store also contains:

• OUTPUT: the rows saved to Dataset, in input order, plus the final run summary.
• RUN_SUMMARY: requested input count; uniqueFetches distinct URL fetch jobs (including rejected URLs); processed saved rows; complete, partial, notFound, errors saved-row counts; retries across distinct fetch jobs; chargedResults recorded result events; unprocessed inputs without a saved row; durationMs; status; chargeLimitReached; and stopReason (COMPLETED, ERROR, ABORTED, or CHARGE_LIMIT).

On graceful cancellation, active HTTP requests stop, and finished rows and counters are saved. A forced kill or unavailable storage can prevent final records from being written. Check the platform run status as well as the records. Duplicate inputs each produce a saved row and a result event if useful.

Pricing and limits

The applied price is $0.001 per Actor start + $0.0018 per complete or partial metadata result. not_found and error rows do not create a metadata-result event. The start event still applies.

This Actor currently uses user-paid platform usage. Apify bills compute, storage operations, and data transfer separately from the event prices. Post-run dataset or storage use can also create normal Apify platform costs. See Actors in Store for the billing model.

Useful results	Start event	Result events	Event charges
0	$0.001	$0	$0.001
1	$0.001	$0.0018	$0.0028
10	$0.001	$0.018	$0.019
100	$0.001	$0.18	$0.181

For 1,000 useful results, use ten runs of at most 100 inputs: modeled event charges are $1.81 in total, plus platform usage. A single 1,000-input run is not supported.

A valid zero-useful-result run still returns error/not-found rows and has the start event and platform usage. A partial row can be billed because it contains usable metadata. A failed or cancelled run can still contain the start event, result events already produced before the stop, and platform usage generated before the stop.

The tested maximum is 100 input items in one run. The capacity test used repeated public URLs to avoid load testing third-party sites, so it proves input/output/storage cardinality up to 100 items rather than 100 different origins at once.

Use cases

Buyer job	Input	Output	Key fields
Audit page metadata	URL list	One row per input item	title, description, canonical, robots
Build link-preview data	URL list	Server-returned social metadata	openGraph, twitter, finalUrl
Preserve duplicate meta values	URL list	Lossless repeated values	openGraph, twitter, metaTags

Scheduling and monitoring

You can run the same input on a schedule with Apify Schedules. This Actor returns a fresh metadata snapshot for each run. It does not compare runs or report changes by itself.

For monitoring, save or export each run and compare the fields you care about, such as title, canonical, openGraph, or twitter.

API, export, and integrations

Use the same JSON input with the Run Actor API. After the run, read the returned Dataset and key-value store IDs.

Dataset results can be exported with Apify Dataset tools. OUTPUT and RUN_SUMMARY are available from the default key-value store for workflows that need a complete run object or counters.

PPE + usage is not eligible for Apify agentic payments. API use requires an Apify account. This Actor does not provide a separate custom MCP server. Tools or agents that can call the Apify API can use the same Actor input and outputs.

Limits and partial results

• Public HTTP(S) pages only. Local, private, link-local, and reserved network addresses are blocked.
• No recursive crawling, sitemap discovery, browser rendering, or JavaScript execution.
• HTML is read as UTF-8. Pages in other encodings may contain incorrect characters.
• HTML is capped at 2 MiB per page. A larger HTML page can return partial with metadata found before the cap.
• Up to 5 redirects. Every redirect target is checked again before connecting.
• JSON-LD, favicon discovery, robots.txt files, and SEO scores are outside this version. robots means the HTML robots meta tag.
• Sites can block direct HTTP requests or return different metadata to browsers. The Actor reports that outcome instead of inventing values.

Troubleshooting

resultStatus is error: check errorType. invalid_url means the input URL format is not supported. blocked_address means the destination resolved to a non-public network address. Temporary network errors, HTTP 429, and HTTP 5xx responses are retried up to maxRetries.

resultStatus is partial: the page returned HTML, but a safety limit stopped full reading. Metadata parsed before the limit is kept. content_too_large means the 2 MiB body cap was reached.

No Open Graph or Twitter fields: the page may not publish those tags in server-returned HTML. This Actor does not run JavaScript, so client-rendered metadata is outside this version.

Run is rejected before fetching: check the input schema and maxTotalChargeUsd. Empty input, more than 100 URLs, or an event-charge cap below the possible result-event total is rejected early.

FAQ

Does it crawl the website?

No. It only fetches the URLs in urls and follows bounded HTTP redirects.

Does it render JavaScript?

No. It reads server-returned HTML with direct HTTP requests.

Are duplicate meta tags preserved?

Yes. When the same key appears more than once, openGraph, twitter, and metaTags keep the values as an array in source order.

Why can metadata be null?

The source page may not publish that tag, or a technical/safety limit may stop full parsing. Check resultStatus, errorType, and RUN_SUMMARY before using nullable fields.

Is every URL guaranteed to work?

No. The destination must be public and reachable with direct HTTP. Sites can block requests, change markup, redirect, or require browser execution.

What does a zero-result run cost?

It still has the $0.001 Actor start event plus user-paid platform usage. No metadata-result event is charged when every row is not_found or error.

Support

Open the Actor's Issues tab in Apify for a bug or feature request. Include the run ID, one non-sensitive example URL, resultStatus, and errorType. Do not send tokens, cookies, private URLs, or account data.