# Website Metadata & SEO Tag Extractor (`fleet261114/website-metadata-extractor`) Actor

Extract title, meta description, canonical, robots, headings, Open Graph, Twitter cards, JSON-LD structured data, hreflang, feeds and favicon from any list of URLs, with a plain list of on-page SEO issues per page. Fast, no browser.

- **URL**: https://apify.com/fleet261114/website-metadata-extractor.md
- **Developed by:** [Fleet](https://apify.com/fleet261114) (community)
- **Categories:** SEO tools, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.40 / 1,000 page results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Metadata & SEO Tag Extractor

Paste a list of page URLs and get, for each page, every tag that search engines and social networks read: **title, meta description, canonical, robots and X-Robots-Tag, headings, Open Graph, Twitter card, JSON-LD structured data (schema.org types), microdata types, hreflang alternates, RSS feeds, favicon**, plus word count, link and image counts, and a plain list of **on-page issues** (missing description, several `<h1>`, canonical pointing elsewhere, noindex, images without alt, invalid JSON-LD, and more).

It fetches each page once with a plain HTTP request (no browser), so it is fast and cheap: thousands of pages per run.

### Use it for

- **SEO audits:** check titles, descriptions, canonicals and headings across a whole site (give it the URL list from a sitemap).
- **Social previews:** see which pages lack `og:image` or a Twitter card before a campaign.
- **Structured data checks:** list the schema.org types (Product, Article, FAQPage, BreadcrumbList...) each page declares and catch broken JSON-LD.
- **Competitor research:** compare how competitors title and describe their pages.
- **Link previews and enrichment:** get title, description, image and favicon for any URL, e.g. for an app or an AI agent.

### Input

| Field | What it does |
|---|---|
| **Page URLs** (required) | One per line, up to 50,000. Each page is fetched once; links are not followed. |
| Return only pages with issues | Only pages with at least one issue are returned (and charged). |
| Include full JSON-LD | Full parsed JSON-LD blocks (default) or only the list of types. |
| Respect robots.txt | Default on: pages disallowed by robots.txt are skipped, listed in `RUN_SUMMARY`, and not charged. |
| Timeout / parallel requests | Defaults 30 s, 10 in total, 2 per website. |

### Output

One item per page that could be read. A real example from a test run (shortened: the full item also lists `h2`, `hreflang` and more):

```json
{
  "url": "https://wordpress.org/plugins/",
  "finalUrl": "https://wordpress.org/plugins/",
  "statusCode": 200,
  "responseTimeMs": 1100,
  "title": "WordPress Plugins | WordPress.org",
  "titleLength": 33,
  "metaDescription": "Choose from thousands of free plugins to build, customize, and enhance your WordPress website.",
  "metaDescriptionLength": 94,
  "metaRobots": "max-image-preview:large",
  "canonicalUrl": "https://wordpress.org/plugins/",
  "lang": "en-US",
  "h1": ["Plugins"],
  "h1Count": 1,
  "openGraph": {
    "title": "WordPress Plugins",
    "description": "Choose from thousands of free plugins to build, customize, and enhance your WordPress website.",
    "site_name": "WordPress.org",
    "type": "website",
    "url": "https://wordpress.org/plugins"
  },
  "twitterCard": {},
  "jsonLdTypes": ["WebSite"],
  "wordCount": 1028,
  "issues": ["Missing og:image"]
}
```

#### The issues list

Issues are factual checks, not opinions: missing `<title>`, title over 60 or under 10 characters, missing meta description or over 160 / under 50 characters, no `<h1>` or more than one, no canonical or a canonical pointing to another URL, `noindex` in meta robots or X-Robots-Tag, no viewport tag, no `lang` attribute, missing `og:title` or `og:image`, images without `alt`, and JSON-LD blocks that are not valid JSON.

### Pricing

Pay per result: **$0.002 per page returned** ($2 per 1,000 pages), with lower prices on Apify's Bronze, Silver and Gold plans, plus Apify's small run-start fee. Pages that cannot be read (network errors, HTTP errors, non-HTML files, robots.txt disallowed), invalid lines and duplicates are **not charged**; they are listed in `RUN_SUMMARY` with the reason. You can set a maximum cost per run; the Actor stops cleanly when it is reached.

### Limits and honest notes

- The Actor reads the HTML the server sends. Pages that build their tags with JavaScript in the browser (some single-page apps) may show fewer tags here than in a browser. Search engines that do not run JavaScript see the same as this Actor.
- Some sites block requests from data centers (bot protection). Such pages are reported as not read and not charged.
- `Person` entries in JSON-LD (author names, emails) are **redacted** on purpose; only `{"@type": "Person", "redacted": true}` is kept. No other personal data is collected.
- Up to 5 MB of HTML is read per page.
- Check sites you have a legitimate reason to check; the defaults are deliberately polite.

### Use it from the API, a schedule or an AI agent

**API.** Start a run and get the results in one call (replace `YOUR_APIFY_TOKEN`):

```bash
curl -X POST "https://api.apify.com/v2/acts/fleet261114~website-metadata-extractor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://www.python.org/", "https://wordpress.org/plugins/"]}'
```

This waits up to 5 minutes. For large lists, start a run with `POST /v2/acts/fleet261114~website-metadata-extractor/runs` and read its dataset when it finishes.

**Whole site.** Get the site's URL list with a sitemap extractor (for example [Sitemap URL Extractor](https://apify.com/fleet261114/sitemap-url-extractor)) and pass the URLs here.

**Schedules and integrations.** Save your input as a task, attach an Apify schedule (e.g. weekly), and send results to Google Sheets, Slack, Make, Zapier or n8n.

**AI agents (MCP).** Add the Actor as a tool in any MCP client (Claude, Cursor and others) through Apify's hosted MCP server:

```
https://mcp.apify.com?tools=fleet261114/website-metadata-extractor
```

The agent sees the input schema and field descriptions and can call the Actor directly. Set a maximum cost per run in the call options to cap spending.

### FAQ

**Does it crawl my whole site?** No. It checks exactly the URLs you give. Feed it a sitemap's URL list to cover a site.

**Why is a tag missing that I see in my browser?** Your page probably adds it with JavaScript after loading. This Actor reads the server's HTML, like most crawlers that don't render JavaScript.

**Why was my page "not read"?** See `RUN_SUMMARY`: the reason is given (timeout, HTTP 403/404, not HTML, robots.txt). You are not charged for it.

**What does a typical run cost?** At $0.002 per page, a 500-page site costs about $1.

### Questions and problems

Open an issue on the Actor's **Issues** tab with the run link.

# Actor input Schema

## `urls` (type: `array`):

One page URL per line (up to 50,000 per run). URLs without a scheme get https://. Each page is fetched once; links are not followed.

## `onlyPagesWithIssues` (type: `boolean`):

Return only pages where at least one on-page issue was found (e.g. missing meta description, several <h1>, noindex). All pages are still checked; you pay only for the results returned.

## `includeJsonLd` (type: `boolean`):

Include the parsed JSON-LD structured data blocks. Off = only the list of schema.org types (smaller output). Person entries are always redacted.

## `respectRobotsTxt` (type: `boolean`):

Skip pages the site's robots.txt disallows for this Actor's user agent (they are listed in RUN_SUMMARY and not charged). Turn off only for your own sites.

## `timeoutSecs` (type: `integer`):

How long to wait for each page before giving up.

## `concurrency` (type: `integer`):

Pages fetched at the same time across all websites.

## `perHostConcurrency` (type: `integer`):

Keeps the load on any one website low.

## Actor input object example

```json
{
  "urls": [
    "https://www.python.org/",
    "https://wordpress.org/plugins/"
  ],
  "onlyPagesWithIssues": false,
  "includeJsonLd": true,
  "respectRobotsTxt": true,
  "timeoutSecs": 30,
  "concurrency": 10,
  "perHostConcurrency": 2
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://www.python.org/",
        "https://wordpress.org/plugins/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fleet261114/website-metadata-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://www.python.org/",
        "https://wordpress.org/plugins/",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("fleet261114/website-metadata-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://www.python.org/",
    "https://wordpress.org/plugins/"
  ]
}' |
apify call fleet261114/website-metadata-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fleet261114/website-metadata-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/HnwcuIVPX9vldV8We/builds/PxGDUQYKaM6hPU1rB/openapi.json