Guardian Articles & Liveblogs Scraper

Fetches Guardian articles and liveblogs — with the full body text — through the Guardian's own Open Platform Content API.

Coverage	2,690,015 items at last check, back to 1899
Returns	headline, standfirst, trail text, byline, full body text, word count, contributor / keyword / tone / series tags, section, pillar, dates, thumbnail
Liveblogs	optional block-by-block update timeline with per-update text, time and contributors
Filtering	free-text search, section, tag, date range, sort order — all applied upstream, not after the fact
Method	HTTP only, one documented JSON API. No browser, no scraping, no login

You need your own API key

Register a free Developer key in about 30 seconds at open-platform.theguardian.com/access. It arrives by email. Paste it into the apiKey input.

Tier	Cost	Limits	Use
Developer	Free	60 calls/min, 500 calls/day, includes article text	Non-commercial only — student work, research, hackathons, non-profit apps
Commercial	Paid	Custom	Everything else, including AI/ML training, text and data mining, and any product derived from Guardian content

This matters, so it is worth being direct about it. The Developer tier is free but explicitly non-commercial. If you are reselling this data, publishing a paid product built on it, or using it to train models, the Commercial key is the tier that applies — the Guardian names those use cases on that page. The actor works identically on either.

Why the API and not a scraper

The Guardian's robots.txt puts ClaudeBot and friends under Disallow: /, its header comment rules out LLM/AI and commercial use in prose, and theguardian.com/license.xml is a machine-readable licence permitting AI use only under a paid subscription. This actor therefore reads their API instead of their website.

That is also simply better data: ~442 URLs are reachable from the news sitemap, versus 2.69 million items here, with real pagination, upstream filtering and an archive reaching back to 1899.

Example input

{
  "apiKey": "your-key-here",
  "queries": ["climate change"],
  "sections": ["environment"],
  "contentTypes": ["article", "liveblog"],
  "dateFrom": "2026-01-01",
  "maxItemsPerQuery": 100,
  "includeBody": true
}

Filters alone, with no search term, are a complete query:

{
  "apiKey": "your-key-here",
  "tags": ["technology/artificialintelligenceai"],
  "orderBy": "newest",
  "maxItemsPerQuery": 200
}

Output

Every row carries the portfolio envelope — _input, _source, _scrapedAt, recordType. Three record types share the dataset:

• ARTICLE / LIVEBLOG — one per item. Same field set; contentType and recordType tell them apart, and a liveblog adds liveblogUpdates.
• SEARCH_SUMMARY — one per query: the API's own total and pages, how many rows came back, how many pages were read, the filters actually applied, and apiCallsUsed so you can see what the run cost against your daily quota.
• ERROR — one per query that failed, so every input maps to at least one row.

Tags arrive from the API as one flat array mixing contributors, keywords, tone and series. They are split into articleContributors, articleKeywords, articleTone and articleSeries, with the untouched array kept as tagsRaw.

Watch your quota

500 calls a day is the binding constraint on a Developer key, and this actor is built around it:

• maxConcurrency defaults to 1 — leave it there. The real limit is 60 calls a minute (the API says so in its own headers, despite the docs saying one per second), so concurrency buys nothing and only produces rate-limit errors faster.
• pageSize is the real lever: at 200 items per call instead of 50, the same 10,000 items cost 50 calls rather than 200.
• Every run logs your key's actual remaining quota, read from the API's response headers — not a local guess, so it accounts for calls the same key spent in earlier runs today.
• Multiple sections or tags are combined into one upstream call, not one call each.
• A run using more than half your daily budget says so in the log.

Known limits

• Articles and liveblogs only. The API also serves video, galleries, audio, interactives and crosswords; they are out of scope here by design, and asking for one returns a named error rather than failing obscurely.
• includeLiveblogUpdates needs liveblog in contentTypes — otherwise there are no updates to return, and the summary row says so.
• Relevance sorting only means something with a search term.
• A single query can only reach 38,000 items. The API refuses any offset past that with an HTTP 400, whatever the reported total says. The actor stops before the boundary and says so on the summary row. Narrow with a date range or a section/tag filter rather than paging deeper.

Anti-bot

None to speak of — this is a documented developer API, not a scrape. A 401 here means a key problem, not a block, and the actor never retries one: rotating a fingerprint cannot fix a rejected credential, and each retry would spend another call from your daily budget.