# WordPress Scraper: Posts, Pages & Categories via REST (`arman-bd/wordpress-blog-scraper`) Actor

Scrape any WordPress site through its public REST API: posts, pages, categories, tags, authors and media. Works on self-hosted and WordPress.com sites alike.

- **URL**: https://apify.com/arman-bd/wordpress-blog-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Social media, Marketing, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.56 / 1,000 post scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## WordPress Scraper: Posts, Pages & Categories via REST

![WordPress Blog Scraper: Posts, pages or any public custom post type from any WordPress site, plain text, author, categories, tags and timestamps](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/wordpress-blog-scraper.jpg)

**WordPress Blog Scraper** pulls posts, pages or any public custom post type from a **WordPress** site through its REST API, title, excerpt, full plain-text content, author, categories, tags, featured image and timestamps.

Roughly 43% of the web runs WordPress, and a stock install publishes its content for programmatic use. This Actor reads that directly: **no browser, no proxies, no login.**

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wordpress-blog-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wordpress-blog-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `site` | Site the record came from |
| `postId`, `slug`, `link` | Identity and the public URL |
| `title` | Post title, entity-decoded |
| `excerpt` | Excerpt as clean plain text |
| `contentPlain` | Full body as clean plain text, HTML stripped, entities decoded |
| `status` | `publish`, and whatever else the site exposes publicly |
| `author`, `authorName` | Author ID, and the display name where the site publishes it |
| `categories`, `tags` | Slug lists, resolved from term IDs |
| `publishedAt`, `modifiedAt` | UTC ISO-8601 timestamps |
| `featuredImage` | Full-size featured image URL |
| `commentCount` | Comment count where the comments route is open, see the caveat below |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, and any site that failed.

### Common use cases

- **Migration and archiving.** Export a whole blog to structured JSON before a replatform.
- **Competitive content analysis.** Track what rivals publish, how often, and under which categories.
- **Multi-site content index.** Ingest dozens of publisher sites into one searchable dataset.
- **Newsletter and aggregation.** Pull the latest posts from a curated list of sites on a schedule.

### Quick start

Two sites, the 100 most recent posts each:

```json
{
 "sites": ["wordpress.org/news", "techcrunch.com"]
}
```

One category, published this year, capped:

```json
{
 "sites": ["wordpress.org/news"],
 "categories": ["releases"],
 "sinceDate": "2026-01-01",
 "maxPostsPerSite": 50
}
```

Pages instead of posts:

```json
{
 "sites": ["wordpress.org/news"],
 "postType": "pages",
 "maxPostsPerSite": 0
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `sites` | array | - | **Required.** Domains, sub-directory installs (`example.com/blog`) or full article URLs. |
| `postType` | string | `posts` | `posts`, `pages`, or any public custom type registered with `show_in_rest`. |
| `categories` | array | `[]` | Category slugs. Resolved to term IDs automatically. `posts` only. |
| `searchQuery` | string | `""` | Full-text search, executed by the site. |
| `sinceDate` | string | `""` | `2026-01-01` or a full ISO-8601 timestamp. |
| `maxPostsPerSite` | integer | `100` | Cap per site. `0` = whole archive. |

`categories`, `searchQuery` and `sinceDate` all run **server-side** and combine with AND. Set `maxPostsPerSite` deliberately: a large publisher can have 250,000+ posts, and `0` means all of them.

### Output example

```json
{
 "site": "https://wordpress.org/news",
 "postId": 21248,
 "title": "WordPress 7.1 Release Candidate 1",
 "slug": "wordpress-7-1-release-candidate-1",
 "excerpt": "The first Release Candidate (\"RC1\") for WordPress 7.1 is ready for download and testing!",
 "contentPlain": "The first Release Candidate (\"RC1\") for WordPress 7.1 is ready…",
 "status": "publish",
 "author": 14679285,
 "authorName": "Benjamin Zekavica",
 "categories": ["development", "general", "releases"],
 "tags": ["7-1", "development", "release"],
 "publishedAt": "2026-08-05T15:37:33Z",
 "modifiedAt": "2026-08-05T15:37:33Z",
 "featuredImage": "https://wordpress.org/news/files/2026/07/wordpress-7-1-RC1.png",
 "commentCount": 0,
 "link": "https://wordpress.org/news/2026/08/wordpress-7-1-release-candidate-1/",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~wordpress-blog-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "sites": ["wordpress.org/news"],
 "categories": ["releases"],
 "maxPostsPerSite": 25
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/wordpress-blog-scraper').call({
 sites: ['wordpress.org/news', 'techcrunch.com'],
 sinceDate: '2026-01-01',
 maxPostsPerSite: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const post of items) console.log(`${post.publishedAt}, ${post.title}`);
```

### Limits and behaviour

- **It does not use `?_embed`, on purpose.** On a site with custom taxonomies, WordPress embeds whole post objects, including the entire `yoast_head` blob, for every related term. Measured on a large publisher, `_embed` returned **32 MB for 25 posts** and blew the request timeout. This Actor requests only the fields it maps and resolves authors, terms and media in batched `include=` lookups instead: the same 100 posts come back in ~1.7 MB and under two seconds.
- **A closed sub-route degrades. It does not fail.** Plenty of publishers return 404 on `/wp/v2/users` or 403 on `/wp/v2/comments`. Those fields become `null`, a warning is logged, and the posts are still saved.
- **`commentCount` is honest.** Core WordPress publishes no comment count on the post object, so it is tallied from the comments collection. Where that route is closed, or holds more comments than can be counted in the page budget, the value is `null` rather than a guess.
- **Bad sites don't kill the run.** A disabled REST API, a security-plugin block or a DNS failure is recorded in `RUN_SUMMARY.failures`; the Actor only errors out if *every* site fails.
- **HTML 404 pages are rejected.** Sites that disable the REST API often answer with the themed 404 page at HTTP 200. The content type is checked, so that is treated as "REST disabled", not as data.
- **Transient errors are retried.** 429, 5xx and network faults get three attempts with exponential backoff and a 30-second timeout.
- **Pagination is read, not guessed.** Page count comes from the `x-wp-totalpages` response header.
- **Public data only.** No authentication, no personal data, no access-control bypass.

### FAQ

**Does it work on WordPress.com sites?** Yes, WordPress.com serves the same `/wp-json/wp/v2/` routes on custom-domain and `*.wordpress.com` sites.

**What if a site disabled the REST API?** It is reported in `RUN_SUMMARY.failures` with the reason, and the run continues with the other sites.

**How do I find a category slug?** Open a category archive on the site, `/category/releases/` means the slug is `releases`. Unknown slugs are logged and skipped.

**Can I scrape a custom post type?** Yes, if it was registered with `show_in_rest: true`. Put its REST base in `postType` (for example `events`).

**Why is `authorName` null on some sites?** Those sites block `/wp/v2/users`, usually via a security plugin, or attribute posts through a plugin such as Co-Authors Plus. The numeric `author` ID is still returned.

**Can I get only new posts?** Run on a schedule with `sinceDate` set to your last run, or diff on `postId` plus `modifiedAt`.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `sites` (type: `array`):

Site domains or URLs. A bare domain ('techcrunch.com'), a sub-directory install ('example.com/blog') or a full article URL are all accepted and normalised automatically.

## `postType` (type: `string`):

Which REST collection to read: 'posts', 'pages', or any public custom post type registered with show_in_rest (for example 'products' or 'events').

## `categories` (type: `array`):

Keep only posts in these categories. Use the slug from the category URL (/category/startups/ → 'startups'). Slugs are resolved to term IDs automatically. Applies to postType 'posts' only.

## `searchQuery` (type: `string`):

Full-text search executed by the site itself across title and content. Leave empty to list everything.

## `sinceDate` (type: `string`):

Only posts published after this date. Accepts a plain date ('2026-01-01') or a full ISO-8601 timestamp. Leave empty for the whole archive.

## `maxPostsPerSite` (type: `integer`):

Cap the number of posts saved per site. Set 0 for the entire archive. note that large publishers have hundreds of thousands of posts.

## Actor input object example

```json
{
  "sites": [
    "techcrunch.com",
    "https://wordpress.org/news/"
  ],
  "postType": "pages",
  "categories": [
    "startups",
    "ai"
  ],
  "searchQuery": "funding round",
  "sinceDate": "2026-01-01",
  "maxPostsPerSite": 100
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sites": [
        "wordpress.org/news",
        "techcrunch.com"
    ],
    "postType": "posts"
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/wordpress-blog-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "sites": [
        "wordpress.org/news",
        "techcrunch.com",
    ],
    "postType": "posts",
}

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/wordpress-blog-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sites": [
    "wordpress.org/news",
    "techcrunch.com"
  ],
  "postType": "posts"
}' |
apify call arman-bd/wordpress-blog-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/wordpress-blog-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/K1cS45QuMJ39jZ7w5/builds/rQ88PuENhEdTu7ZG0/openapi.json