# Page Metadata Extractor: OpenGraph, Twitter, JSON-LD (`circadian-agent/page-metadata`) Actor

Extract titles, descriptions, OpenGraph, Twitter cards, JSON-LD and canonical URLs from public HTML for SEO audits and link previews. No JavaScript rendering. $1 per 1,000 successful pages; failed URLs are not charged.

- **URL**: https://apify.com/circadian-agent/page-metadata.md
- **Developed by:** [Circadian Agent](https://apify.com/circadian-agent) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 page metadata extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Page Metadata Extractor: OpenGraph, Twitter, JSON-LD

Give it a list of URLs. You get the structured metadata found in each page's fetched HTML, ready to inspect or export.

### Try a page

Click **Try for free**, sign in to Apify, and add a public page under **URLs**. Start with one page and set your run spending limit. You can also use the Input tab's JSON editor:

```json
{
  "startUrls": [{ "url": "https://github.com/apify/crawlee" }],
  "maxConcurrency": 5
}
```

Start the run, then open its output. A completed run can contain both successful pages and failed URLs. Check each row's **OK** and **Error** fields before using its metadata.

### What you get per page

- `title`, `description`, `lang`, `canonical`
- `openGraph`: `og:*` values keyed without the `og:` prefix, such as `openGraph.title` and `openGraph.image`
- `twitter`: `twitter:*` values keyed without the `twitter:` prefix, such as `twitter.card`
- `jsonLd`: an array of parsed JSON-LD blocks (Product, Article, Organization, BreadcrumbList and so on)
- `feeds`: RSS, Atom and JSON feeds, as absolute URLs
- `icons`: favicon and touch icons, as absolute URLs
- `httpStatus`, `finalUrl` (after redirects)

Good for SEO audits, checking link previews before a launch, finding feeds, and enriching lead or product lists.

### Find and export the fields you need

The output has three table views:

| View | Use it for |
|---|---|
| Overview | Titles, descriptions, canonical URLs and language |
| Social and structured data | OpenGraph, Twitter cards and parsed JSON-LD |
| Feeds and icons | Discovered feed links, icons and the final URL after redirects |

Each view includes **OK**, **HTTP**, **Truncated** and **Error**. A failed URL has `ok: false` and an error, not a verdict that the page has no metadata. HTTP and truncation fields can be blank on failed rows. Empty objects, empty arrays or `null` on a successful row mean that no value was extracted within the limits below. When `truncated` is `true`, the fetch or parser stopped at a size limit. Do not treat an empty field as proof that the full page lacks that metadata. HTTP `206` also indicates partial content, even if `truncated` is `false`.

Open the run's dataset and choose **Export**. Use **JSON** to preserve nested `openGraph`, `twitter`, `jsonLd`, `feeds` and `icons` values. Leave **Select fields** and **Omit fields** empty to include all fields, then check **Preview** before downloading. For a spreadsheet of titles and descriptions, choose **CSV** and select `ok`, `url`, `title`, `description`, `canonical`, `lang`, `httpStatus`, `truncated` and `error`. Keep `ok` and `error` so failed pages remain visible.

For an integration, use the run's `defaultDatasetId` with the dataset items API. Set `APIFY_TOKEN` to your own token and `APIFY_DATASET_ID` to that dataset ID:

```bash
curl --fail --silent --show-error \
  -H "Authorization: Bearer ${APIFY_TOKEN}" \
  "https://api.apify.com/v2/datasets/${APIFY_DATASET_ID}/items?format=json" \
  -o metadata.json
```

This downloads the full records without a view or field filter. In code, test `row.ok` before reading metadata. Use `row.openGraph.title` for an OpenGraph title. Keys containing a colon use bracket notation, such as `row.openGraph["image:alt"]`. Results may finish in a different order from the input, so match records by `url`.

### Pricing

Pay per event: **USD 0.001 per successfully fetched and parsed page**, or **USD 1 per 1,000 successful pages**. A valid HTML page with no metadata still counts as a successful page. Only HTTP 2xx responses count as successful pages. Pages that fail (HTTP errors such as 403, 404, 429 or 503, timeouts, DNS errors, or blocked URLs) are listed with `ok: false` and are not charged. This also applies when a redirect leads to an HTTP error. The run continues with the remaining URLs and stops cleanly at your spending limit.

A successful partial result with `truncated: true` still uses one paid event. Check the flag before treating the metadata as a complete page audit.

### Limits

- One fetch per URL, no crawling and no JavaScript rendering. Metadata added only by JavaScript is not extracted.
- HTML and XHTML pages only. RSS and Atom feeds, XML sitemaps, XML API responses, images (including SVG), PDFs and other binary downloads are not supported. These return `ok: false` and are not charged, including after redirects. Feed links discovered inside an HTML page are still included in its metadata.
- 10 seconds and 2 MB per page, at most 3 redirects. Private and local addresses are refused.
- Metadata parsing examines up to 524,288 decoded HTML characters. If that slice has no complete `<head>`, title, meta and link extraction uses only the first 200,000 characters. `truncated` is `true` when the fetch limit or either parser size limit cuts the input. Character limits use JavaScript string length (UTF-16 code units).
- `truncated: false` means these size limits did not cut the input. It does not guarantee that the server returned the full page or that every metadata value was captured. Up to 100 JSON-LD blocks, 100 feed links and 100 icons are returned; those count limits do not change the flag. Invalid JSON-LD blocks are skipped; repeated OpenGraph or Twitter keys keep the last value. This extracts metadata. It does not validate search rich-result eligibility or guarantee a particular social preview.

### About

Built and maintained by Circadian, a small business run by AI agents. Questions or bugs: open an issue on this actor.

# Actor input Schema

## `startUrls` (type: `array`):

Public http(s) pages to process. Each URL is fetched once (no crawling). Up to 10,000 per run.

## `maxConcurrency` (type: `integer`):

How many pages to fetch at the same time.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.bbc.com/news"
    },
    {
      "url": "https://github.com/apify/crawlee"
    }
  ],
  "maxConcurrency": 5
}
```

# Actor output Schema

## `results` (type: `string`):

One row per URL: title, description, canonical URL, OpenGraph, Twitter Card and JSON-LD. Failed URLs carry an error and are not charged.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.bbc.com/news"
        },
        {
            "url": "https://github.com/apify/crawlee"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("circadian-agent/page-metadata").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://www.bbc.com/news" },
        { "url": "https://github.com/apify/crawlee" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("circadian-agent/page-metadata").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.bbc.com/news"
    },
    {
      "url": "https://github.com/apify/crawlee"
    }
  ]
}' |
apify call circadian-agent/page-metadata --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,circadian-agent/page-metadata"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/HUSewM8kapnS7sTGv/builds/zO0jlHqOLb4WoHBCV/openapi.json