# Website Metadata & Open Graph Scraper (`cliqtomedia/website-metadata-open-graph-scraper`) Actor

Extract title, description, canonical, robots, Open Graph, Twitter Card and raw meta tags from up to 100 public URLs.

- **URL**: https://apify.com/cliqtomedia/website-metadata-open-graph-scraper.md
- **Developed by:** [Cliqto Media](https://apify.com/cliqtomedia) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.80 / 1,000 metadata results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Extract metadata from public HTTP and HTTPS pages. Get the page title, meta description, canonical URL, robots meta tag, Open Graph fields, Twitter Card fields, and named meta tags.

Add 1–100 URLs. The Actor makes direct HTTP requests, returns one Dataset row for each input item, keeps repeated metadata values, and resolves relative metadata URLs. It does not crawl links or run JavaScript.

![Website Metadata & Open Graph Scraper: URLs to metadata](https://api.apify.com/v2/key-value-stores/xPZ8kFHoVanPFnygM/records/HERO?signature=9508fef89b8ea42bf1e537cb383768)

### Table of contents

- [What it does](#what-it-does)
- [What data you get](#what-data-you-get)
- [Quick start](#quick-start)
- [Input](#input)
- [Input example](#input-example)
- [Output example](#output-example)
- [Output fields](#output-fields)
- [Pricing and limits](#pricing-and-limits)
- [Use cases](#use-cases)
- [Scheduling and monitoring](#scheduling-and-monitoring)
- [API, export, and integrations](#api-export-and-integrations)
- [Limits and partial results](#limits-and-partial-results)
- [Troubleshooting](#troubleshooting)
- [FAQ](#faq)
- [Support](#support)

### What it does

The Actor fetches only the URLs you provide and reads metadata from the returned HTML. It follows bounded HTTP redirects, but it does not discover or crawl other pages.

Exact duplicate input URLs share one network fetch, while each input item still gets its own output row. If the same meta key appears more than once, the output keeps all values in source order.

### What data you get

Each processed input can include:

- HTML title and meta description.
- Canonical URL and robots meta value.
- All `og:*` Open Graph values.
- All `twitter:*` Twitter Card values.
- Named/property meta tags in `metaTags`, with URL-like values resolved.
- HTTP status, final URL, content type, timing, attempts, and result status.
- Safe error fields for invalid, blocked, missing, or failed URLs.

Relative canonical, Open Graph URL/image, and Twitter URL/image values are resolved against the final page URL. Missing metadata stays `null` or an empty object.

### Quick start

1. Open the **Input** tab.
2. Add one public URL to `urls`, for example `https://example.com/`.
3. Keep the default request settings for a normal run.
4. Set `maxTotalChargeUsd` high enough for the possible event charges, then click **Start**.
5. Open **Dataset** and check `resultStatus`, `title`, and the metadata fields.

For one possible useful result, allow at least **$0.0028** in event charges. For 100 possible useful results, allow at least **$0.181**. Platform usage is billed separately by Apify for this Actor.

### Input

| Field | Plain name | Type | Required | Runtime default | UI prefill | Allowed values | Effect | Recommendation |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `urls` | URLs | array of strings | Yes | None | `https://example.com/`, `https://apify.com/` | 1–100 HTTP(S) URLs, max 2048 characters each | Pages to fetch. Exact duplicates share one fetch but keep one output row per input item. | Start with a small URL list, then increase only when the source pages respond normally. |
| `maxConcurrency` | Concurrent requests | integer | No | `10` | None | 1–20 | Maximum URLs fetched at the same time. | Keep `10` unless the target sites are slow or rate-limited. |
| `requestTimeoutSecs` | Request timeout | integer | No | `20` | None | 3–60 | Maximum time for one HTTP attempt. | Increase only for known slow pages. |
| `maxRetries` | Retries | integer | No | `1` | None | 0–3 | Retries temporary network errors, HTTP 429, and HTTP 5xx. | Keep `1` for normal runs. |

Empty `urls`, more than 100 items, unknown input fields, and wrong types are rejected before fetch work starts. API input `{}` is invalid; the UI prefill is a small demo, not a runtime URL default.

`maxRetries: 1` allows one retry after the first failed attempt. A URL can make more requests when it redirects (up to 5 redirects). The timeout limits each HTTP attempt; retries and redirects can increase total time. Lower concurrency for rate-limited sites. The platform run timeout can stop a long list before every item is processed.

### Input example

```json
{
  "urls": [
    "https://example.com/",
    "https://www.w3.org/"
  ],
  "maxConcurrency": 2,
  "requestTimeoutSecs": 20,
  "maxRetries": 1
}
```

This run fetches the two supplied pages and returns two Dataset rows when both items are processed.

### Output example

This is one real row from `https://example.com/`, trimmed only to remove timing fields:

```json
{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "ok": true,
  "status": 200,
  "title": "Example Domain",
  "description": null,
  "canonical": null,
  "robots": null,
  "openGraph": {},
  "twitter": {},
  "metaTags": {
    "viewport": "width=device-width,initial-scale=1"
  },
  "resultStatus": "complete",
  "errorType": null,
  "errorMessage": null
}
```

### Output fields

| Field | Type | Can be empty? | Example | Meaning |
| --- | --- | --- | --- | --- |
| `url` | string | No | `https://example.com/` | Input URL; credential userinfo is redacted in rejected URLs. |
| `finalUrl` | string or null | Yes | `https://example.com/` | URL after redirects, when a connection was made. |
| `ok` | boolean | No | `true` | `true` for complete or partial HTML results. |
| `status` | integer or null | Yes | `200` | HTTP status code when available. |
| `contentType` | string or null | Yes | `text/html` | Response Content-Type when available. |
| `responseTimeMs` | integer or null | Yes | `120` | Total URL fetch time, including redirects and retries. |
| `attempts` | integer | No | `1` | Total HTTP attempts for this URL. |
| `fetchedAt` | ISO string | No | `2026-10-05T12:00:00Z` | Time the row was created. |
| `title` | string or null | Yes | `Example Domain` | First HTML `<title>`. |
| `description` | string or null | Yes | `null` | First meta description; repeats remain in `metaTags.description`. |
| `canonical` | string or null | Yes | `https://site.com/page` | First canonical link, resolved to an absolute URL. |
| `robots` | string or null | Yes | `index,follow` | First robots meta value. Repeats remain in `metaTags.robots`. |
| `openGraph` | object | Yes | `{"image":["https://site.com/a.png","https://site.com/b.png"]}` | All `og:*` values. Repeated keys become arrays; URL-like values are resolved. |
| `twitter` | object | Yes | `{"card":"summary"}` | All `twitter:*` values. Repeated keys become arrays; URL-like values are resolved. |
| `metaTags` | object | Yes | `{"description":["First","Second"]}` | Named/property meta tags. Repeated keys become arrays. |
| `resultStatus` | string | No | `complete` | `complete`, `partial`, `not_found`, or `error`. |
| `errorType` | string or null | Yes | `not_found` | Stable category such as `invalid_url`, `blocked_address`, `not_found`, or `content_too_large`. |
| `errorMessage` | string or null | Yes | `The page was not found.` | Safe error detail for this item. |

The default key-value store also contains:

- `OUTPUT`: the rows saved to Dataset, in input order, plus the final run summary.
- `RUN_SUMMARY`: `requested` input count; `uniqueFetches` distinct URL fetch jobs (including rejected URLs); `processed` saved rows; `complete`, `partial`, `notFound`, `errors` saved-row counts; `retries` across distinct fetch jobs; `chargedResults` recorded result events; `unprocessed` inputs without a saved row; `durationMs`; `status`; `chargeLimitReached`; and `stopReason` (`COMPLETED`, `ERROR`, `ABORTED`, or `CHARGE_LIMIT`).

On graceful cancellation, active HTTP requests stop, and finished rows and counters are saved. A forced kill or unavailable storage can prevent final records from being written. Check the platform run status as well as the records. Duplicate inputs each produce a saved row and a result event if useful.

### Pricing and limits

The applied price is **$0.001 per Actor start + $0.0018 per complete or partial metadata result**. `not_found` and `error` rows do not create a `metadata-result` event. The start event still applies.

This Actor currently uses user-paid platform usage. Apify bills compute, storage operations, and data transfer separately from the event prices. Post-run dataset or storage use can also create normal Apify platform costs. See [Actors in Store](https://docs.apify.com/actors/running/actors-in-store) for the billing model.

| Useful results | Start event | Result events | Event charges |
| ---: | ---: | ---: | ---: |
| 0 | $0.001 | $0 | $0.001 |
| 1 | $0.001 | $0.0018 | $0.0028 |
| 10 | $0.001 | $0.018 | $0.019 |
| 100 | $0.001 | $0.18 | $0.181 |

For 1,000 useful results, use ten runs of at most 100 inputs: modeled event charges are $1.81 in total, plus platform usage. A single 1,000-input run is not supported.

A valid zero-useful-result run still returns error/not-found rows and has the start event and platform usage. A partial row can be billed because it contains usable metadata. A failed or cancelled run can still contain the start event, result events already produced before the stop, and platform usage generated before the stop.

The tested maximum is **100 input items in one run**. The capacity test used repeated public URLs to avoid load testing third-party sites, so it proves input/output/storage cardinality up to 100 items rather than 100 different origins at once.

### Use cases

| Buyer job | Input | Output | Key fields |
| --- | --- | --- | --- |
| Audit page metadata | URL list | One row per input item | `title`, `description`, `canonical`, `robots` |
| Build link-preview data | URL list | Server-returned social metadata | `openGraph`, `twitter`, `finalUrl` |
| Preserve duplicate meta values | URL list | Lossless repeated values | `openGraph`, `twitter`, `metaTags` |

### Scheduling and monitoring

You can run the same input on a schedule with [Apify Schedules](https://docs.apify.com/platform/schedules). This Actor returns a fresh metadata snapshot for each run. It does not compare runs or report changes by itself.

For monitoring, save or export each run and compare the fields you care about, such as `title`, `canonical`, `openGraph`, or `twitter`.

### API, export, and integrations

Use the same JSON input with the [Run Actor API](https://docs.apify.com/api/v2/act-runs-post). After the run, read the returned Dataset and key-value store IDs.

Dataset results can be exported with Apify Dataset tools. `OUTPUT` and `RUN_SUMMARY` are available from the default key-value store for workflows that need a complete run object or counters.

PPE + usage is not eligible for Apify agentic payments. API use requires an Apify account. This Actor does not provide a separate custom MCP server. Tools or agents that can call the Apify API can use the same Actor input and outputs.

### Limits and partial results

- Public HTTP(S) pages only. Local, private, link-local, and reserved network addresses are blocked.
- No recursive crawling, sitemap discovery, browser rendering, or JavaScript execution.
- HTML is read as UTF-8. Pages in other encodings may contain incorrect characters.
- HTML is capped at 2 MiB per page. A larger HTML page can return `partial` with metadata found before the cap.
- Up to 5 redirects. Every redirect target is checked again before connecting.
- JSON-LD, favicon discovery, robots.txt files, and SEO scores are outside this version. `robots` means the HTML robots meta tag.
- Sites can block direct HTTP requests or return different metadata to browsers. The Actor reports that outcome instead of inventing values.

### Troubleshooting

**`resultStatus` is `error`:** check `errorType`. `invalid_url` means the input URL format is not supported. `blocked_address` means the destination resolved to a non-public network address. Temporary network errors, HTTP 429, and HTTP 5xx responses are retried up to `maxRetries`.

**`resultStatus` is `partial`:** the page returned HTML, but a safety limit stopped full reading. Metadata parsed before the limit is kept. `content_too_large` means the 2 MiB body cap was reached.

**No Open Graph or Twitter fields:** the page may not publish those tags in server-returned HTML. This Actor does not run JavaScript, so client-rendered metadata is outside this version.

**Run is rejected before fetching:** check the input schema and `maxTotalChargeUsd`. Empty input, more than 100 URLs, or an event-charge cap below the possible result-event total is rejected early.

### FAQ

#### Does it crawl the website?

No. It only fetches the URLs in `urls` and follows bounded HTTP redirects.

#### Does it render JavaScript?

No. It reads server-returned HTML with direct HTTP requests.

#### Are duplicate meta tags preserved?

Yes. When the same key appears more than once, `openGraph`, `twitter`, and `metaTags` keep the values as an array in source order.

#### Why can metadata be null?

The source page may not publish that tag, or a technical/safety limit may stop full parsing. Check `resultStatus`, `errorType`, and `RUN_SUMMARY` before using nullable fields.

#### Is every URL guaranteed to work?

No. The destination must be public and reachable with direct HTTP. Sites can block requests, change markup, redirect, or require browser execution.

#### What does a zero-result run cost?

It still has the `$0.001` Actor start event plus user-paid platform usage. No `metadata-result` event is charged when every row is `not_found` or `error`.

### Support

Open the Actor's **Issues** tab in Apify for a bug or feature request. Include the run ID, one non-sensitive example URL, `resultStatus`, and `errorType`. Do not send tokens, cookies, private URLs, or account data.

# Actor input Schema

## `urls` (type: `array`):

Add 1–100 public HTTP(S) page URLs. Exact duplicates share one network fetch but still return one row per input item.

## `maxConcurrency` (type: `integer`):

Maximum number of URLs fetched at the same time.

## `requestTimeoutSecs` (type: `integer`):

Maximum time for one HTTP attempt, in seconds.

## `maxRetries` (type: `integer`):

Retries for temporary network errors, HTTP 429 and HTTP 5xx responses.

## Actor input object example

```json
{
  "urls": [
    "https://example.com/",
    "https://apify.com/"
  ],
  "maxConcurrency": 10,
  "requestTimeoutSecs": 20,
  "maxRetries": 1
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `output` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com/",
        "https://apify.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("cliqtomedia/website-metadata-open-graph-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://example.com/",
        "https://apify.com/",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("cliqtomedia/website-metadata-open-graph-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com/",
    "https://apify.com/"
  ]
}' |
apify call cliqtomedia/website-metadata-open-graph-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cliqtomedia/website-metadata-open-graph-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/DS2vhlbqL1S4edqLB/builds/9fxQblK1rBIsUmPLQ/openapi.json