Website Metadata Extractor Pro

Extract comprehensive website metadata from any URL in bulk: title, description, Open Graph, Twitter Cards, JSON-LD, favicon, canonical, robots, sitemap, language, and charset.

Pricing

from $3.00 / 1,000 result items

Rating

0.0

(0)

Developer

Harsh

Actor stats

Bookmarked

Total users

Monthly active users

8 days ago

Last modified

Input

Field	Type	Required	Default	Description
`urls`	`string[]`	Yes	—	URLs to extract metadata from
`includeTwitterCards`	`boolean`	No	`true`	Include Twitter Card meta tags
`maxRequestsPerMinute`	`integer`	No	`60`	Rate limit for HTTP requests
`maxRequestRetries`	`integer`	No	`3`	Retries per failed request
`requestHandlerTimeoutSecs`	`integer`	No	`30`	Per-request timeout in seconds
`proxyConfiguration`	`object`	No	—	Optional Apify Proxy settings

Output (per URL)

Field	Description
`url`	Input URL (resolved after redirects)
`title`	Page title (`og:title` or `<title>`)
`description`	Page description (`og:description` or meta description)
`openGraph`	Object with all `og:*` meta properties
`twitterCards`	Object with all `twitter:*` meta properties
`jsonLd`	Array of parsed JSON-LD structured data objects
`favicon`	Favicon URL (resolved to absolute)
`canonical`	Canonical link URL
`robots`	Robots meta tag content
`sitemap`	Sitemap link URL from `<link rel="sitemap">`
`language`	Page language from `<html lang>`
`charset`	Document charset
`error`	Error message if extraction failed (`null` on success)

How it works

Fetches each URL with Crawlee CheerioCrawler (retries, rate limiting, optional proxy)
Parses HTML with Cheerio
Extracts Open Graph, Twitter Card, JSON-LD, canonical, favicon, robots, sitemap, language, and charset
Resolves relative URLs to absolute against the page URL
Pushes one dataset item per URL with graceful per-URL error handling

Run locally

npm install
apify run

Example input (storage/key_value_stores/default/INPUT.json):

{
    "urls": ["https://apify.com", "https://github.com"],
    "includeTwitterCards": true,
    "maxRequestsPerMinute": 60,
    "maxRequestRetries": 3,
    "requestHandlerTimeoutSecs": 30
}

See examples/input.json and examples/output-sample.json for more examples.

Deploy to Apify

apify login
apify push

Configure pay-per-event pricing with apify-default-dataset-item at $0.003 per result in the Apify Console monetization settings.

Resources

Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags

haketa/bulk-metadata-extractor

Extract on-page metadata from thousands of URLs at once: title, meta description, Open Graph & Twitter Card tags, canonical, favicon, hreflang, RSS feed and schema.org JSON-LD structured data. Fast, keyless bulk SEO metadata and structured-data extractor.