# Bulk SEO Metadata Extractor: Title, Meta, Open Graph & JSON-LD (`hillcountrybots/bulk-seo-metadata-extractor`) Actor

Title, meta description, canonical, robots, hreflang, Open Graph, H1 and JSON-LD types for a URL list or a whole sitemap. $2 per 1,000 pages, failures free.

- **URL**: https://apify.com/hillcountrybots/bulk-seo-metadata-extractor.md
- **Developed by:** [Hill Country Bots](https://apify.com/hillcountrybots) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 page extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk SEO Metadata Extractor: Title, Meta, Open Graph & JSON-LD

Pull the **SEO metadata** of hundreds of pages in one run. This **bulk meta tag extractor** fetches each page once and returns the **title**, **meta description**, **canonical**, **meta robots and X-Robots-Tag**, **hreflang**, html **lang**, every **H1**, **Open Graph** and **Twitter card** tags, **JSON-LD schema types**, viewport, favicon and word count. Paste a URL list or point it at a **sitemap**. You get one clean row per page, ready for CSV or Google Sheets. **Plain HTTP, respects robots.txt, no login, no personal data.** **$2 per 1,000 pages. 404s, non-HTML files, timeouts, robots.txt skips and invalid inputs are free.**

### What does this SEO metadata extractor do?

- **URLs mode:** extract metadata for exactly the URLs you paste
- **Sitemap mode:** give a website or sitemap URL. It reads robots.txt, sitemap indexes and gzip, then extracts every listed page up to `maxPages`
- Returns title and meta description **with lengths**, so too-long or missing tags are easy to sort
- Flags `isIndexable` (noindex in meta robots, googlebot or X-Robots-Tag) and `canonicalIsSelf`
- Lists **schema.org types** from JSON-LD (Organization, Product, Article, FAQPage, BreadcrumbList and so on), plus how many JSON-LD blocks are invalid JSON
- Writes a `RUN_SUMMARY` with counts of missing titles, descriptions, canonicals, H1s and og:images, and the top **duplicate titles**

### Who is it for?

- **SEO agencies:** a site-wide title and meta description sheet for audits, migrations and client reports
- **Content and social teams:** check Open Graph and Twitter card tags before sharing links
- **Developers and AI agents:** link-preview data and structured data types as stable JSON through the API or MCP

### Quick start

1. Paste page URLs. The form is prefilled with three, so **Start** works right away.
2. For a whole site, switch **Mode** to **Every page in the sitemap** and give the homepage.
3. Click **Start**, then export JSON, CSV or Excel, or use the **Open Graph & Twitter** view.

#### Sample output

From a real run on **November 12, 2026, at 8:04 AM CT** (America/Chicago), URLs mode, 10 inputs: 7 pages extracted in under 3 seconds, and the 404, the JSON file and the invalid URL came back as free error rows.

| URL | Title (length) | Desc. length | Canonical | H1s | JSON-LD types | hreflang |
|---|---|---|---|---|---|---|
| https://wordpress.org/ | Blog Tool, Publishing Platform, and CMS | WordPress.org (55) | 90 | self | 1 | Organization, ImageObject, WebSite, WebPage | 100+ |
| https://apify.com/ | Apify: Marketplace of ready-to-run tools for AI | 143 | self | 1 | Organization, PostalAddress, ContactPoint, WebSite | 0 |
| https://web.dev/ | web.dev (7) | 66 | self | **0** | Article, Organization, WebSite, SearchAction | 44 |
| https://www.python.org/ | Welcome to Python.org (21) | 52 | **missing** | **5** | WebSite, SearchAction | 0 |
| https://httpbin.org/status/404 | (error HTTP_404, free) | | | | | |

#### Example inputs

Metadata for a list of pages:

```json
{
  "mode": "urls",
  "startUrls": ["https://wordpress.org/", "https://apify.com/", "https://web.dev/"]
}
```

Every page in a site's sitemap (up to 500):

```json
{
  "mode": "sitemap",
  "startUrls": ["https://www.sitemaps.org/"],
  "maxPages": 500
}
```

Only blog posts from a sitemap:

```json
{
  "mode": "sitemap",
  "startUrls": ["https://example.com/sitemap.xml"],
  "includePatterns": ["https://example.com/blog/*"],
  "maxPages": 1000
}
```

### Input

| Field | Type | Default | What it does |
|---|---|---|---|
| `startUrls` | array of strings | (required, prefilled) | URLs mode: pages. Sitemap mode: websites or sitemap URLs. Private and internal addresses are refused (free). |
| `mode` | `urls`, `sitemap` | `urls` | Pasted URLs only, or every page in the sitemap. |
| `maxPages` | integer | `1000` | Spend cap: pages fetched in the run. |
| `includePatterns` / `excludePatterns` | arrays | (none) | `*` globs or `/regex/` for sitemap URLs. |
| `respectRobotsTxt` | boolean | `true` | Disallowed pages are not fetched (free error row). |
| `maxConcurrency` | integer 1-30 | `10` | Pages in flight in total. |
| `maxConcurrencyPerHost` | integer 1-5 | `3` | Pages in flight per site. |
| `requestTimeoutSecs` | integer | `20` | Seconds before a page is marked TIMEOUT (retried once). |

### Output example

```json
{
  "url": "https://wordpress.org/",
  "finalUrl": "https://wordpress.org/",
  "statusCode": 200,
  "title": "Blog Tool, Publishing Platform, and CMS | WordPress.org",
  "titleLength": 55,
  "metaDescription": "Open source software which you can use to easily create a beautiful website, blog, or app.",
  "metaDescriptionLength": 90,
  "canonical": "https://wordpress.org/",
  "canonicalIsSelf": true,
  "metaRobots": "max-image-preview:large",
  "isIndexable": true,
  "htmlLang": "en-US",
  "hreflang": [{"hreflang": "af-za", "href": "https://af.wordpress.org/"}],
  "h1Count": 1,
  "jsonLdTypes": ["Organization", "ImageObject", "WebSite", "WebPage"],
  "status": "ok",
  "errorCode": null
}
```

Each row also has `h1`, `h1All`, `ogTitle`, `ogDescription`, `ogImage`, `ogUrl`, `ogType`, `ogSiteName`, `twitterCard`, `twitterTitle`, `twitterDescription`, `twitterImage`, `jsonLdBlocks`, `jsonLdInvalidBlocks`, `metaViewport`, `favicon`, `wordCount`, `redirectCount`, `responseTimeMs` and `sourceSitemap`. Missing tags are `null` (lengths and counts are `0`), so the CSV columns stay the same on every row.

A page that could not be extracted is **not charged**:

```json
{"url": "https://httpbin.org/status/404", "statusCode": 404, "status": "error", "errorCode": "HTTP_404", "errorMessage": "HTTP 404"}
```

#### For AI agents and integrations

- Branch on `status`: `ok` = one extracted HTML page (the billable event), `error` = free.
- `errorCode` is one of `INVALID_URL`, `PRIVATE_URL`, `EMPTY_INPUT`, `EMPTY_SITEMAP`, `ROBOTS_DISALLOWED`, `HTTP_<code>`, `NOT_HTML`, `DNS_NOT_FOUND`, `CONNECTION_REFUSED`, `TLS_ERROR`, `TIMEOUT`, `NETWORK_ERROR`, `TOO_MANY_REDIRECTS`, `REDIRECT_LOOP`, `UNSAFE_REDIRECT`.

### Use it from the API or MCP

- **Actor ID:** `hillcountrybots/bulk-seo-metadata-extractor`

```bash
curl -X POST "https://api.apify.com/v2/acts/hillcountrybots~bulk-seo-metadata-extractor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":["https://wordpress.org/","https://apify.com/"]}'
```

- **MCP (Claude, Cursor, ChatGPT and other MCP clients):** add `https://mcp.apify.com?tools=hillcountrybots/bulk-seo-metadata-extractor`.

### How much does it cost?

**$2 per 1,000 pages** ($0.002 per `page-extracted` event, one per HTML page that answered 2xx and was parsed). No start fee. You pay nothing for:

- invalid or private inputs
- 4xx and 5xx pages, non-HTML files, timeouts and network errors
- pages skipped by robots.txt, filters or the cap
- the run summary

Example: a 500-page sitemap costs $1.00. Cap spend with `maxPages` or the run's max charge setting.

### Limits

- Plain HTTP only. Tags that JavaScript adds after load (some single-page apps) are not seen.
- It reports what the page says. It does not score pages; for scores and fixes use the [SEO Audit Tool](https://apify.com/hillcountrybots/seo-audit).
- `hreflang` keeps the first 100 alternates; `hreflangCount` has the full number.

### FAQ

**Can I get every page on a site?** Yes, with sitemap mode. If a page is not in the sitemap, paste it in URLs mode.

**Does it check structured data validity?** It lists the schema.org types and counts JSON-LD blocks that are not valid JSON. It does not run Google's rich result test.

**Does it collect personal data?** No. It reads public page metadata only.

**Does it follow redirects?** Yes, up to 10. `finalUrl` and `redirectCount` show where the page ended up.

### More tools from Hill Country Bots

See everything we make at [hillcountrybots.com](https://hillcountrybots.com).

- [SEO Audit Tool](https://apify.com/hillcountrybots/seo-audit): 0-100 SEO score per page, top 5 fixes and a client-ready HTML report. $2 per 1,000 pages, failures free.
- [Sitemap URL Extractor](https://apify.com/hillcountrybots/sitemap-url-extractor): every URL from a site's XML sitemaps. $0.50 per 1,000 URLs.
- [Bulk PageSpeed Insights Test](https://apify.com/hillcountrybots/bulk-pagespeed-checker): Lighthouse scores and Core Web Vitals for many URLs. $3 per 1,000 audits, failures free.
- [Tech Stack Detector](https://apify.com/hillcountrybots/tech-stack-detector): CMS, analytics, CDN and frameworks for many sites. $0.50 per 1,000 sites.
- [Bulk Domain Availability Checker](https://apify.com/hillcountrybots/domain-availability-checker): available or taken, from official registry RDAP. $1 per 1,000 domains.

### Support

Missing a tag you need? Open an issue on the **Issues** tab with the run ID and the URL. We reply within 2 business days (US Central time). Built and maintained by **Hill Country Bots**.

# Actor input Schema

## `startUrls` (type: `array`):

Required. One per line. URLs mode: every page you want metadata for. Sitemap mode: a website (its sitemap is found through robots.txt or /sitemap.xml) or a direct sitemap URL. Private, local and internal addresses are refused for free.

## `mode` (type: `string`):

urls = extract metadata for exactly the URLs you pasted. sitemap = read each site's XML sitemap (sitemap indexes and gzip included) and extract every page it lists, up to the page cap.

## `maxPages` (type: `integer`):

Total pages fetched in this run. Only successfully extracted HTML pages are charged. The run's own max-charge setting is also respected.

## `includePatterns` (type: `array`):

Optional globs with \* (e.g. https://example.com/blog/\*) or /regex/. Sitemap URLs that do not match are skipped.

## `excludePatterns` (type: `array`):

Optional globs or /regex/, e.g. */tag/* or */page/*.

## `respectRobotsTxt` (type: `boolean`):

On (recommended): pages disallowed by the site's robots.txt are not fetched and are returned as free error rows (ROBOTS_DISALLOWED).

## `maxConcurrency` (type: `integer`):

Pages fetched in parallel across all hosts (1-30).

## `maxConcurrencyPerHost` (type: `integer`):

Pages fetched in parallel from any single host (1-5). Keep this low to be polite.

## `requestTimeoutSecs` (type: `integer`):

Seconds to wait for a page before it is marked TIMEOUT (retried once).

## Actor input object example

```json
{
  "startUrls": [
    "https://crawlee.dev/",
    "https://www.wikipedia.org/",
    "https://www.sitemaps.org/protocol.html"
  ],
  "mode": "urls",
  "maxPages": 50,
  "includePatterns": [],
  "excludePatterns": [],
  "respectRobotsTxt": true,
  "maxConcurrency": 10,
  "maxConcurrencyPerHost": 3,
  "requestTimeoutSecs": 20
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset with one row per page: title, meta description, canonical, robots, hreflang, H1, Open Graph, Twitter card and JSON-LD types, plus free error rows.

## `summary` (type: `string`):

JSON run summary: pages extracted and failed, and how many pages miss a title, description, canonical, H1 or Open Graph image, plus duplicate titles. Key: RUN_SUMMARY.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawlee.dev/",
        "https://www.wikipedia.org/",
        "https://www.sitemaps.org/protocol.html"
    ],
    "maxPages": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("hillcountrybots/bulk-seo-metadata-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        "https://crawlee.dev/",
        "https://www.wikipedia.org/",
        "https://www.sitemaps.org/protocol.html",
    ],
    "maxPages": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("hillcountrybots/bulk-seo-metadata-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawlee.dev/",
    "https://www.wikipedia.org/",
    "https://www.sitemaps.org/protocol.html"
  ],
  "maxPages": 50
}' |
apify call hillcountrybots/bulk-seo-metadata-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hillcountrybots/bulk-seo-metadata-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/lTxDKhRQR7gyja7Y0/builds/NjI63afd7QM6bdq2o/openapi.json