WordPress Scraper - Posts, Pages & Authors

Scrape any WordPress site's content in bulk through its public REST API. Extract post title, URL, publish and modified dates, author, categories, tags, excerpt, full text and word count to CSV/JSON. No API key, no login.

---

What does the WordPress Scraper do?

This Actor turns WordPress data into a structured dataset. You point it at one site, full text or several sites, metadata only, it walks the result pages one after another, and it writes one clean row per post into your dataset — ready to export as JSON, CSV or Excel, or to pull straight from the Apify API.

WordPress ships a public REST API at /wp-json/wp/v2/ and leaves it on by default, which is why roughly two in five websites expose their whole archive as JSON. The Actor reads posts straight from it and resolves category, tag and author IDs to names using the site's own taxonomy endpoints, so the export carries readable labels instead of bare numbers. Pagination is followed automatically until it runs out of results, hits your page limit or hits your Max items cap, whichever comes first. Every row is de-duplicated across the whole run, so you are never billed twice for the same post.

There is no API key, no login and no browser involved. That keeps runs fast and cheap, and it means you can schedule the Actor without worrying about credentials expiring.

Who is it for?

• SEO and content teams auditing a competitor's whole archive.
• AI and NLP teams building text corpora from public articles.
• Agencies inventorying a client's content before a migration.
• Market researchers tracking what an industry publisher covers.
• Developers mirroring or syndicating a site's content.

Use cases

• Export a publisher's entire archive with dates, authors and categories.
• Measure publishing cadence and word count over time.
• Find which topics and tags a competitor covers most.
• Build a clean article corpus for embedding or classification.
• Inventory every post and page before replatforming a site.

Why use this WordPress Scraper?

• 🔑 Keyless — no account, no API token, no cookies to paste.
• 📦 24 fields per post — everything the source exposes, already typed.
• 📄 Real pagination — it walks page after page instead of returning the first screen.
• 🎯 Precise caps — Max items stops the run exactly where you want it, so the bill is predictable.
• 📊 Export anywhere — JSON, CSV, Excel or HTML, plus the Apify API and integrations.
• 💸 Pay per result — you pay for rows you actually receive, with no platform fees to calculate.

What data can you extract?

Every run produces one row per post, with these fields:

Field	Type	Description
siteDomain	string	Domain of the WordPress site this row came from
postId	string	WordPress post ID
title	string	Post title with HTML stripped
slug	string	URL slug of the post
url	string	Direct link to the post
contentType	string	Whether the row is a post or a page
status	string	Publication status reported by the API
publishedAt	string	ISO timestamp of publication in GMT
modifiedAt	string	ISO timestamp of the last edit in GMT
authorId	string	Author ID as published by the site
authorName	string	Author display name when the site exposes its users endpoint
categoryNames	string	Category names, separated by a pipe
tagNames	string	Tag names, separated by a pipe
categoryIds	string	Category IDs, separated by a pipe
tagIds	string	Tag IDs, separated by a pipe
excerpt	string	Excerpt with HTML stripped
content	string	Full article text with HTML stripped
wordCount	number	Words in the article text
commentStatus	string	Whether comments are open or closed
format	string	Post format, e.g. standard, video or gallery
isSticky	boolean	Whether the post is pinned to the front page
featuredMediaId	string	ID of the featured image when one is set
page	number	API page the post appeared on
scrapedAt	string	ISO timestamp of extraction

Output example

{
  "siteDomain": "wordpress.org/news",
  "postId": "17205",
  "title": "Episode 77: Let\u2019s Talk About Data Liberation",
  "slug": "episode-77-lets-talk-about-data-liberation",
  "url": "https://wordpress.org/news/2024/04/episode-77-lets-talk-about-data-liberation/",
  "contentType": "post",
  "status": "publish",
  "publishedAt": "2024-04-15T12:00:00Z",
  "modifiedAt": "2024-04-12T20:47:15Z",
  "authorId": "20187494",
  "authorName": "Brett McSherry",
  "categoryNames": "Podcast",
  "tagNames": "wp-briefing",
  "categoryIds": "420",
  "tagIds": "339",
  "excerpt": "Explore the WordPress Data Liberation project in this exclusive behind-the-scenes episode discussing WordPress migrations. Joining us is WordPress Executive\u2026",
  "content": "Explore the WordPress Data Liberation project in this exclusive behind-the-scenes episode discussing WordPress migrations. Joining us is WordPress Executive Director Josepha Haden Chomphosy, along\u2026",
  "wordCount": 7028,
  "commentStatus": "closed",
  "format": "standard",
  "isSticky": false,
  "featuredMediaId": null,
  "page": 2,
  "scrapedAt": "2026-09-19T11:22:03.517Z"
}

How to use

Option A — one site, full text

{
  "includeContent": true,
  "maxItems": 2000,
  "maxPagesPerSite": 20,
  "siteUrls": [
    "https://techcrunch.com"
  ]
}

1. Open the Actor and fill in the first field.
2. Set the page limit and Max items to bound the run.
3. Click Start, then export from the Output tab.

Option B — several sites, metadata only

{
  "contentType": "posts",
  "includeContent": false,
  "maxItems": 5000,
  "maxPagesPerSite": 50,
  "searchQuery": "ai",
  "siteUrls": [
    "https://techcrunch.com",
    "https://www.rollingstone.com"
  ]
}

Everything in the first field is processed independently, so you can batch several targets into one run and split them apart afterwards.

Input parameters

Parameter	Type	Default	Description
siteUrls	array	–	Home pages of the WordPress sites to scrape, e.
contentType	string	posts	Which collection to export: blog posts or static pages.
searchQuery	string	–	Restrict to content matching this term, using the site's own WordPress search.
includeContent	boolean	true	Include the full article text with HTML stripped.
maxPagesPerSite	integer	5	How many result pages to walk for each site.
maxItems	integer	500	Stop after this many posts.
maxConcurrency	integer	2	Parallel requests.
proxyConfiguration	object	{"useApifyProxy": true}	Proxy used to fetch pages.

Tips for best results

• Paste the site's home page — the Actor appends the REST API path for you.
• The API returns at most 100 posts per page; asking for more is rejected by WordPress itself.
• Switch Content type to pages to export static pages instead of blog posts.
• Search query maps to WordPress's own search, so you can pull just the posts that mention a term.
• Turn off Include full content for a far smaller export when you only need metadata.
• wordCount is computed from the stripped text, which makes it comparable across sites.
• Category and tag names come from the site's taxonomy endpoints, fetched once per site rather than embedded in every post — that is what keeps a 100-post page around 3 MB instead of 120 MB.
• Many sites disable the users endpoint for security, so authorName can be empty while authorId is filled.
• A site with the REST API switched off returns no rows and the log says so explicitly.
• Pair with the WordPress Plugin Directory Scraper to study the wider WordPress ecosystem.

Integrations

Send results straight into the tools you already use: Google Sheets, Slack, Zapier, Make, Airtable or any Webhook. You can also schedule the Actor to run hourly, daily or weekly and have each run append to the same dataset, which is how you build a price or availability history rather than a one-off snapshot.

API usage

Run the Actor and collect results from any language. Replace <YOUR_TOKEN> with your Apify API token.

cURL

curl -X POST "https://api.apify.com/v2/acts/logiover~wordpress-site-content-scraper/run-sync-get-dataset-items?token=<YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"includeContent": true, "maxItems": 2000, "maxPagesPerSite": 20, "siteUrls": ["https://techcrunch.com"]}'

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_TOKEN>' });
const run = await client.actor('logiover/wordpress-site-content-scraper').call({"includeContent": true, "maxItems": 2000, "maxPagesPerSite": 20, "siteUrls": ["https://techcrunch.com"]});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient('<YOUR_TOKEN>')
run = client.actor('logiover/wordpress-site-content-scraper').call(run_input={"includeContent": true, "maxItems": 2000, "maxPagesPerSite": 20, "siteUrls": ["https://techcrunch.com"]})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

Use with AI agents (MCP)

This Actor is available through the Apify MCP server, so an AI agent can call it as a tool. Point your agent at https://mcp.apify.com and it can run the WordPress Scraper on demand — for example: "Pull the first 500 posts from WordPress and summarise what you find." The agent receives the same structured rows you would get from the UI.

FAQ

Do I need a WordPress account or API key?

No. The Actor reads publicly available data only. There is nothing to authenticate and no credentials to rotate.

How many posts can I get in one run?

As many as the source exposes. Raise the page limit and Max items together; the run stops at whichever limit it reaches first.

Why did I get fewer rows than I asked for?

The source ran out of posts. That is normal for narrow searches — broaden the query or add more targets to one run.

Are results de-duplicated?

Yes. Each post is emitted once per run, even when it appears on several pages, so you are never billed twice for the same record.

Why are some fields empty?

WordPress does not publish every attribute for every post. Empty means the source did not supply it, not that extraction failed.

What export formats are supported?

JSON, CSV, Excel, HTML and RSS from the Output tab, plus the Apify API and any integration you connect.

How fast is it?

It is pure HTTP with no browser, so a page of results typically takes a second or two. Raise Max concurrency carefully — the source rate-limits aggressive crawling.

Can I schedule it?

Yes. Use the Apify scheduler to run it on any interval and append each run to the same dataset for time-series analysis.

Does it work behind a proxy?

It uses Apify Proxy automatically. You can switch groups or supply your own proxies in Proxy configuration.

How often does the data change?

WordPress updates continuously. Re-run whenever you need current data; the Actor always reads the live source, never a cache.

Is the output schema stable?

Yes. Field names and types are fixed, so downstream pipelines will not break between runs.

What if the source changes its format?

Open an issue on the Issues tab and it gets fixed. The Actor is actively maintained.

Is it legal?

This Actor reads only publicly available data from WordPress — the same content any visitor or client sees without logging in. It does not bypass authentication, does not collect private data and does not attempt to defeat access controls. You are responsible for how you use the output: respect the source's terms of service, applicable copyright, and data-protection law such as GDPR where personal data is involved. Scraping public data is generally lawful in the EU and the US, but the responsibility for the downstream use of that data sits with you.

Related scrapers

• WordPress Plugin Scraper — Ecosystem and market data
• Website Text & Markdown — Any site, not just WordPress
• Website SEO Audit — On-page SEO for the same domain
• Sitemap to URL Crawler — Full URL inventory