# Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags (`haketa/bulk-metadata-extractor`) Actor Extract on-page metadata from thousands of URLs at once: title, meta description, Open Graph & Twitter Card tags, canonical, favicon, hreflang, RSS feed and schema.org JSON-LD structured data. Fast, keyless bulk SEO metadata and structured-data extractor. - **URL**: https://apify.com/haketa/bulk-metadata-extractor.md - **Developed by:** [Haketa](https://apify.com/haketa) (community) - **Categories:** SEO tools, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.60 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags at Scale Extract **on‑page metadata from thousands of URLs at once.** Paste a list and get, for each page: **title, meta description, canonical, Open Graph & Twitter Card tags, favicon, hreflang, RSS feed and schema.org JSON‑LD structured data.** Fast, **keyless** and built for scale — the metadata layer you need for **SEO audits, link previews, structured‑data checks and content intelligence.** --- ### Why this Actor? Every page's `` is packed with signals — how it appears in search, how it unfurls on social, what language versions exist and what structured data it exposes. Reading all of that by hand, page by page, is painful. This Actor pulls it in **bulk** and returns a clean, comparable table: - **SEO tags** — title, description, canonical, robots, viewport, keywords, author. - **Open Graph & Twitter Card** — everything that controls how a link looks when shared. - **Icons & feeds** — favicon, apple‑touch‑icon, web manifest, RSS/Atom feed. - **Internationalisation** — every `hreflang` alternate. - **Structured data** — all schema.org JSON‑LD blocks and the exact types they declare. One run turns a URL list into a full metadata + structured‑data inventory. --- ### What you can do with it - **SEO audits** — check titles, descriptions, canonicals and structured data across a whole site or list. - **Link previews / unfurling** — build rich cards from OG/Twitter tags for chat, CMS and social tools. - **Structured‑data checks** — see which pages expose schema.org (Product, Article, FAQ, Recipe…) and which don't. - **CMS & site migrations** — snapshot metadata before/after to catch regressions. - **Content & competitor intelligence** — map how pages describe themselves and what schema they use. - **Data enrichment** — add title, description, image and site name to any list of URLs. --- ### What you get — output fields One record per URL: | Field | Description | | --- | --- | | `url` / `finalUrl` | Input URL and URL after redirects | | `statusCode` | HTTP status | | `title` | Page title | | `metaDescription` | Meta description | | `canonical` | Canonical URL | | `lang` / `charset` | Language and charset | | `robots` / `viewport` / `themeColor` | Common meta tags | | `author` / `keywords` | Author and keywords meta | | `ogTitle` … `ogLocale` | Open Graph: title, description, image, type, site name, url, locale | | `twitterCard` … `twitterCreator` | Twitter Card: card, title, description, image, site, creator | | `favicon` / `appleTouchIcon` / `manifestUrl` | Icons and web manifest | | `feedUrl` | RSS/Atom feed | | `hreflang` | All hreflang language alternates | | `schemaTypes` | schema.org types found (e.g. `Product, BreadcrumbList`) | | `jsonLdCount` | Number of JSON‑LD blocks | | `structuredData` | Raw JSON‑LD (when enabled) | | `error` | Populated if the page couldn't be read | | `scrapedAt` | ISO timestamp | All URLs are absolute (favicon, images and canonical are resolved against the page). --- ### Example output ```json { "url": "https://www.theverge.com", "statusCode": "200", "title": "The Verge", "metaDescription": "The Verge is about technology and how it makes us feel.", "canonical": "https://www.theverge.com/", "lang": "en-US", "ogTitle": "The Verge", "ogType": "website", "ogSiteName": "The Verge", "ogImage": "https://.../verge-social.png", "twitterCard": "summary_large_image", "favicon": "https://www.theverge.com/favicon.ico", "feedUrl": "https://www.theverge.com/rss/index.xml", "hreflang": "", "schemaTypes": "NewsMediaOrganization, WebSite", "jsonLdCount": "2", "scrapedAt": "2026-07-10T14:00:00.000Z" } ```` *** ### Input | Field | Type | Default | Description | | --- | --- | --- | --- | | `urls` | array | — | **Required.** Page URLs to read. | | `includeJsonLd` | boolean | `true` | Include raw JSON‑LD in `structuredData`. | | `maxConcurrency` | integer | `20` | URLs processed in parallel. | | `proxyConfiguration` | object | off | Apify Proxy, if some targets block direct requests. | #### Basic run ```json { "urls": ["https://www.theverge.com", "https://github.com", "https://stripe.com"] } ``` #### Types only (smaller output) ```json { "urls": ["https://example.com/page-1", "https://example.com/page-2"], "includeJsonLd": false } ``` URLs without a scheme are accepted (`example.com` becomes `https://example.com`). *** ### Use cases in detail #### 1. Site‑wide SEO audit Feed every URL from your sitemap and get titles, descriptions, canonicals and robots directives in one table — instantly see missing or duplicate tags. #### 2. Structured‑data coverage `schemaTypes` shows exactly which pages expose Product, Article, FAQPage, Recipe, BreadcrumbList and more — find pages missing the schema they should have for rich results. #### 3. Link previews and unfurling Use the OG and Twitter fields (`ogTitle`, `ogImage`, `twitterCard`…) to render rich link cards in your app, CMS or chat integration. #### 4. Migration QA Snapshot metadata before a migration and again after, then diff the two exports to catch dropped titles, canonicals or structured data. #### 5. Competitive & content intelligence See how competitors write their titles/descriptions and which schema types they invest in across their key pages. #### 6. Enrich a URL list Turn a plain list of URLs into records with title, description, image and site name — ready for a CRM, spreadsheet or dataset. *** ### How to use it 1. Click **Try for free**. 2. Paste your URLs. 3. (Optional) Turn off `includeJsonLd` or adjust concurrency. 4. Click **Start**. 5. Export as **JSON, CSV, Excel** or **HTML table**, or via the Apify API. *** ### Calling from the API ```bash curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "urls": ["https://www.theverge.com", "https://stripe.com"] }' ``` Fetch the results (JSON or CSV): ```bash curl "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs/last/dataset/items?token=YOUR_APIFY_TOKEN&format=csv" ``` *** ### What it reads - **Standard meta** — title, description, keywords, author, robots, viewport, theme‑color, charset, html lang. - **Canonical & alternates** — `rel=canonical` and every `hreflang` alternate. - **Open Graph** — all `og:*` properties (title, description, image, type, site\_name, url, locale). - **Twitter Card** — all `twitter:*` tags. - **Icons & app** — favicon, apple‑touch‑icon, web app manifest. - **Feeds** — RSS/Atom `link` tags. - **Structured data** — every `application/ld+json` block, with all schema.org `@type`s (including nested `@graph`). *** ### Tips for the best results - **Feed a sitemap's URLs** for a full‑site audit in one run. - **Filter `schemaTypes` is empty** to find pages missing structured data. - **Compare `title` vs `ogTitle`** and `metaDescription` vs `ogDescription` to catch inconsistencies. - **Turn off `includeJsonLd`** when you only need tags and types — the output gets much smaller. - **Enable proxy** if a set of targets rate‑limits direct requests. *** ### Frequently asked questions **Does it render JavaScript?** It reads the served HTML (which is where metadata, OG/Twitter tags and JSON‑LD almost always live). This keeps it fast and cheap; SPA content injected purely client‑side into the body isn't required for metadata. **Does it get all JSON‑LD types?** Yes — it parses every JSON‑LD block and collects all `@type`s, including those nested in `@graph`. **Can I pass URLs without https://?** Yes — a bare domain or path is upgraded to `https://` automatically. **How large can the structured data be?** Very large blocks are captured (capped per record to keep exports manageable). Turn `includeJsonLd` off if you only need the types. **How many URLs can it handle?** From a handful to millions, processed in parallel. **In which format can I export?** JSON, CSV, Excel, HTML table, or via the Apify API. **Do I need an API key?** No. Just paste your URLs and run. *** ### Notes This Actor reads publicly available HTML metadata from the URLs you provide. Use it on pages you have a legitimate reason to read and in line with each site's terms. *** ### Support Need extra signals (e.g. microdata/RDFa, full meta dump, per‑image details)? Open an issue from the Actor's page and describe what you need. # Actor input Schema ## `urls` (type: `array`): The page URLs to extract metadata from. Paste a list — handles thousands to millions. ## `includeJsonLd` (type: `boolean`): Include the full schema.org JSON-LD blocks in a structuredData field. Turn off to keep only the detected schema types (smaller output). ## `maxConcurrency` (type: `integer`): How many URLs to process in parallel. ## `proxyConfiguration` (type: `object`): Apify Proxy. Off by default; enable if some targets block direct requests. ## Actor input object example ```json { "urls": [ "https://www.theverge.com", "https://github.com", "https://stripe.com" ], "includeJsonLd": true, "maxConcurrency": 20, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `url` (type: `string`): The input URL ## `finalUrl` (type: `string`): URL after redirects ## `statusCode` (type: `string`): HTTP status code ## `title` (type: `string`): Page title ## `metaDescription` (type: `string`): Meta description ## `canonical` (type: `string`): Canonical URL ## `lang` (type: `string`): HTML lang ## `charset` (type: `string`): Charset ## `robots` (type: `string`): Robots meta ## `viewport` (type: `string`): Viewport meta ## `themeColor` (type: `string`): Theme color ## `author` (type: `string`): Author meta ## `keywords` (type: `string`): Keywords meta ## `ogTitle` (type: `string`): Open Graph title ## `ogDescription` (type: `string`): Open Graph description ## `ogImage` (type: `string`): Open Graph image ## `ogType` (type: `string`): Open Graph type ## `ogSiteName` (type: `string`): Open Graph site name ## `ogUrl` (type: `string`): Open Graph URL ## `ogLocale` (type: `string`): Open Graph locale ## `twitterCard` (type: `string`): Twitter card type ## `twitterTitle` (type: `string`): Twitter title ## `twitterDescription` (type: `string`): Twitter description ## `twitterImage` (type: `string`): Twitter image ## `twitterSite` (type: `string`): Twitter site ## `twitterCreator` (type: `string`): Twitter creator ## `favicon` (type: `string`): Favicon URL ## `appleTouchIcon` (type: `string`): Apple touch icon URL ## `manifestUrl` (type: `string`): Web app manifest URL ## `feedUrl` (type: `string`): RSS/Atom feed URL ## `hreflang` (type: `string`): hreflang languages ## `schemaTypes` (type: `string`): schema.org types found ## `jsonLdCount` (type: `string`): Number of JSON-LD blocks ## `structuredData` (type: `string`): Raw JSON-LD ## `error` (type: `string`): Error, if any ## `scrapedAt` (type: `string`): Timestamp # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "https://www.theverge.com", "https://github.com", "https://stripe.com" ], "maxConcurrency": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("haketa/bulk-metadata-extractor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [ "https://www.theverge.com", "https://github.com", "https://stripe.com", ], "maxConcurrency": 20, } # Run the Actor and wait for it to finish run = client.actor("haketa/bulk-metadata-extractor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "https://www.theverge.com", "https://github.com", "https://stripe.com" ], "maxConcurrency": 20 }' | apify call haketa/bulk-metadata-extractor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=haketa/bulk-metadata-extractor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags", "description": "Extract on-page metadata from thousands of URLs at once: title, meta description, Open Graph & Twitter Card tags, canonical, favicon, hreflang, RSS feed and schema.org JSON-LD structured data. Fast, keyless bulk SEO metadata and structured-data extractor.", "version": "0.1", "x-build-id": "SgTm2BiDr8VIkVbNE" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/haketa~bulk-metadata-extractor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-haketa-bulk-metadata-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/haketa~bulk-metadata-extractor/runs": { "post": { "operationId": "runs-sync-haketa-bulk-metadata-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/haketa~bulk-metadata-extractor/run-sync": { "post": { "operationId": "run-sync-haketa-bulk-metadata-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "urls" ], "properties": { "urls": { "title": "URLs", "type": "array", "description": "The page URLs to extract metadata from. Paste a list — handles thousands to millions.", "items": { "type": "string" } }, "includeJsonLd": { "title": "Include raw JSON-LD", "type": "boolean", "description": "Include the full schema.org JSON-LD blocks in a structuredData field. Turn off to keep only the detected schema types (smaller output).", "default": true }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "maximum": 100, "type": "integer", "description": "How many URLs to process in parallel.", "default": 20 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Apify Proxy. Off by default; enable if some targets block direct requests.", "default": { "useApifyProxy": false } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```