Google News Scraper API — real publisher URLs, full text, sentiment, deep archive

Google News Suite  •  7 Actors, one codebase, one article billed once
Google News Scraper API ➤ You are here	Google News API JSON News Feed with Real Article URLs	News Monitoring Scraper Track Brands, Competitors & Topics
News Archive Scraper Historical News Past the 100-Item Limit	Breaking News API Live Headlines Feed as JSON	Finance News Scraper API Market & Ticker News as JSON
Crypto News Scraper API Bitcoin & Altcoin News as JSON

More from this account: YouTube Transcript Suite · Website Contact & Email Suite · Career Site & ATS Jobs Suite · Keyword Research Suite · Shopify Store Intelligence Suite · eBay Data Suite · Amazon Reviews Suite · Reddit · Meta Ad Library · Vinted

Scrape Google News into clean JSON. Every article comes back with the publisher's real URL already resolved — not a news.google.com/rss/articles/CBMi… redirect blob — plus the headline, outlet and published date. Turn on full text and you also get the article body, author, summary, image and canonical URL. Every article is scored for sentiment, free.

Pay per article. Failed runs and empty runs cost nothing.

// Input
{ "queries": ["tesla"], "maxItems": 25 }

// Output row
{
  "title": "Tesla's true believers are starting to question their faith in Elon Musk",
  "source": "The Washington Post",
  "publishedAt": "2026-08-18T10:00:01.000Z",
  "publisherUrl": "https://www.washingtonpost.com/technology/2026/08/18/teslas-true-believers…",
  "publisherDomain": "washingtonpost.com",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMiuwFBVV95cUxNNVpvQkFlV1lB…",
  "resolveStatus": "ok"
}

---

Why this one

1. The URL is the real article, not a Google redirect

Google's RSS feed hands you an encoded CBMi… token. It is not a URL you can fetch, store, deduplicate, or hand to a model. The old trick of base64-decoding it stopped working — of 99 feed items checked on 2026-08-19, zero decoded to a URL.

This Actor resolves it properly. Measured: 500 of 500 articles resolved to a real publisher domain in a single run (and 100/100 in another). Not a sample — every row.

Several competing Actors either return the redirect blob on default settings, hide resolution behind a flag, or bill it as a separate paid event. Here it is on by default and included in the article price. If a URL genuinely cannot be resolved you get publisherUrl: null and a resolveStatus telling you why — never a guessed URL.

2. Break the 100-article ceiling

A Google News feed returns at most ~100 items, whatever you ask for — num= and start= are ignored. Every scraper built on that feed inherits the ceiling.

Set archiveDays and this one splits your query into consecutive date windows and fetches each separately:

Run	Articles returned
One plain query	101
Same query, archiveDays: 14	500 (capped by maxItems, not by Google)

Measured 2026-08-19: 500 unique articles, 500/500 resolved, 196 distinct outlets, in 31 seconds. Coverage reaches back years — verified returning full windows for 2015, 2018, 2020, 2022 and 2025.

Honest caveat: archive coverage is query-specific, not a clean date cutoff. Some query/window pairs return nothing at all — tesla returns 0 results for June 2024 on repeated attempts while apple and election return 100 for that exact window. That is Google's index, not a fault in the Actor, and the diagnostics row reports what each window returned so you can see it.

3. Full article text, and you only pay when it arrives

Turn on fetchArticleText to fetch each publisher page and extract the body, author, canonical URL and image.

News publishers block and paywall aggressively, so here is the measured breakdown from one real 100-article run rather than a marketing number:

textStatus	Rows	What it means
ok	50	Full body text returned
blocked	30	Publisher refused the request (401/403)
too-short	15	Page fetched, no article body found
paywalled	3	Publisher declares the content not free
fetch-failed	2	Connection failed or timed out

You are charged for the 50, not the 100. Paywalled, blocked and empty extractions are free, and every row carries an explicit textStatus so you always know which you got. Reuters, Politico, NYT and Barron's are among the outlets that commonly block — the row still arrives with headline, outlet, date and resolved URL.

4. Five ways in, one run

Combine any of these in a single run; results are spread evenly across them, not drained from whichever you listed first:

• queries — keywords, with Google operators: "exact phrase", tesla OR rivian, tesla -musk, intitle:tesla, site:reuters.com
• topics — WORLD, BUSINESS, TECHNOLOGY, SPORTS, SCIENCE, HEALTH, ENTERTAINMENT, NATION, or an opaque topic ID from a Google News URL
• locations — local editions by place name: London, New York, Berlin
• publishers — restrict to named outlets: reuters.com, bbc.co.uk
• includeTopHeadlines — Google's front page for your language and country

Any language and country edition: language: "de", country: "DE", and so on.

5. Sentiment that is actually there

Every competing listing measured advertises a sentiment field. None of them emits one. This does, on every article, at no extra charge.

It uses VADER — the standard lexicon for short news and social text — not a hand-rolled word list, so it handles negation properly ("Tesla is not doing badly" scores positive rather than negative). You get sentiment (positive / negative / neutral), sentimentScore (-1 to +1) and sentimentBasis telling you what was scored: the full article body when you have it, otherwise headline plus summary, otherwise the headline.

The honest limit: VADER's lexicon is English. Scoring a German or Japanese headline against it would produce a confident number derived from almost no matched words — so non-English articles return sentiment: null with sentimentBasis: "unsupported-language" instead. A null you can filter on beats a number you cannot trust.

6. Collapse syndicated duplicates

One wire story runs across dozens of outlets. Set dedupeSyndicated and you get one row per story, with the other outlets attached as republishedBy[] — so you pay once for a story instead of forty times. Matching is on a normalized headline, so re-headlined copy ("Tesla recalls 12,000 cars" vs "12,000 cars recalled by Tesla") still collapses.

---

Input

Field	Type	Default	Description
queries	string[]	—	Search keywords. Google operators supported.
topics	string[]	—	BUSINESS, TECHNOLOGY, … or an opaque topic ID.
locations	string[]	—	Local editions by place name.
publishers	string[]	—	Restrict to these domains.
publisherQuery	string	—	Keyword filter applied to publishers.
includeTopHeadlines	boolean	false	Add Google's front-page headlines.
archiveDays	integer	0	Days back to reach. 0 = one latest-news pull.
windowDays	integer	1	Size of each archive window.
maxItems	integer	100	Hard cap on billable rows.
maxCostUsd	string	—	Optional spend ceiling; the run stops cleanly before it.
language	string	en	en, de, ja, pt-BR, …
country	string	US	US, GB, DE, IN, SE, …
resolvePublisherUrls	boolean	true	Resolve the Google redirect.
fetchArticleText	boolean	false	Fetch body text, author, canonical URL.
dedupeSyndicated	boolean	false	One row per story.
includeDuplicates	boolean	false	Keep collapsed copies, tagged.
concurrency	integer	20	Parallel requests.
proxy	object	off	Optional. Not needed — see below.

At least one of queries, topics, locations, publishers or includeTopHeadlines is required. Anything else fails immediately with a message that names the fix, and costs nothing.

Output

Field fill rates, measured over a real 100-article run:

Field	Filled	Notes
title, publishedAt, source, sourceUrl	100%	Straight from the feed
publisherUrl, publisherDomain	100%	Resolved
imageUrl	65%	Needs fetchArticleText
snippet	63%	Needs fetchArticleText — see note
canonicalUrl	61%	Needs fetchArticleText
articlePublishedAt	52%	Publisher's own timestamp, often more precise than Google's
articleText	50%	See the textStatus table above
author	47%	Many outlets simply do not publish one

On snippet: Google's RSS feed does not actually carry a summary — its description field is just the headline and outlet wrapped in markup (measured: 0 of 101 items on a search feed had anything more). Rather than echo the title back at you, this Actor leaves snippet null there and fills it from the publisher's own summary when fetchArticleText is on. Other scrapers pass Google's markup through as if it were a summary.

Every row also carries resolveStatus, textStatus, matchedInput, inputType, window, language, country and scrapedAt. Rows collapsed by dedupe carry duplicateOfId; canonical rows carry republishedBy[].

Each run also emits one free diagnostics row — items collected, resolve rate, text extraction rate, windows used, errors, estimated cost — and a free error row for any input that failed, so a partial run is still actionable and never silently short.

Pricing

Three events, all tiered by your Apify plan. Per 1,000:

Event	What it is	FREE	GOLD	DIAMOND
article	One article: headline, outlet, date, sentiment, image	$7.00	$3.50	$3.10
publisher-url	Google's redirect resolved to the real article URL	$0.80	$0.50	$0.40
article-text	One full body successfully extracted	$1.80	$1.40	$1.00
	per resolved article	$7.80	$4.00	$3.50

Every event bills only on success:

• A resolution that fails is free. The row still arrives with publisherUrl: null and a resolveStatus. Set resolvePublisherUrls: false and you are never charged for it at all.
• Text that is paywalled, blocked or empty is free. Measured: on a 40-article run, 19 extractions failed and 19 were not billed.
• Sentiment is free. No event, no charge.
• No run-start fee. An empty run costs exactly $0.00 — 5 of the 11 competing actors charge one, up to $0.09 per run before a single row is returned.
• Set maxCostUsd to cap any run; all three events count against it.

Typical run: 1,000 articles ≈ $3.10–7.00 depending on your plan tier. The same 1,000 with full text on returns ~500 bodies and adds ~$0.50–1.00.

Platform compute on top of that is negligible and we tuned it deliberately: a measured 100-article run with full text enabled costs $0.017 per 1,000 articles in Apify compute, because this Actor is pure HTTP — no headless browser — and runs at 1 GB rather than the 4 GB most Actors default to.

Use it from anywhere

API — start a run and get the rows back in one call:

curl -X POST "https://api.apify.com/v2/acts/vonsensey~google-news-scraper-api/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"queries":["your topic"],"maxItems":50}'

Python

from apify_client import ApifyClient
client = ApifyClient("YOUR_TOKEN")
run = client.actor("vonsensey/google-news-scraper-api").call(
    run_input={"queries": ["tesla"], "archiveDays": 30, "maxItems": 500}
)
for a in client.dataset(run["defaultDatasetId"]).iterate_items():
    if a.get("type") == "article":
        print(a["publishedAt"], a["source"], a["publisherUrl"])

n8n / Make / Zapier — use the Apify integration, pick this Actor, and map the dataset output. Schedule it and you have a monitoring feed.

Feeding a model? Turn on fetchArticleText and use articleText plus publisherUrl for citations. textStatus tells you which rows have a real body, so you can filter before embedding rather than indexing empty strings.

Notes

• No proxy needed. Google News is reachable directly from Apify, which is why this is fast and cheap. The proxy input exists only if you need a specific country of origin.
• Speed: 500 articles with resolved URLs in ~31 seconds; 100 articles with full text in ~36 seconds.
• Reliability: failures never kill a run. A bad query, a dead publisher or a blocked page produces an error or a status field, and the run completes.

Legal

This is an unofficial tool and is not affiliated with, endorsed by, or sponsored by Google. "Google" and "Google News" are trademarks of Google LLC. It reads publicly available Google News RSS feeds and public publisher pages — no login, no paywall circumvention. Article text is provided for monitoring, indexing and analysis; you are responsible for respecting publisher copyright in how you use it.

More scrapers from Blackcube

• eBay Scraper API — sold prices, listings, sellers
• Amazon Reviews Scraper API — reviews and ratings
• Reddit Scraper — posts and comments
• Facebook Ads Library Scraper — ad intelligence
• Website Contact & Email Extractor — B2B contacts
• YouTube Transcript Scraper — transcripts for RAG

Issues and feature requests go in the Issues tab — first response within 24 hours.

Run it without configuring anything — Scrape Google News for a search term, a ready-made example you can start as-is or copy.

Use cases

• Brand and competitor monitoring. Track your own name and a rival's in one run and see both in the same dataset.
• Feed a model or a dashboard. Clean JSON with the real publisher URL, headline, source and publication time — no scraping of your own required.
• Build a historical dataset. Walk a date window to reach far more than the ~100 items a single news query returns.
• Run it on a schedule. News is a feed, not a one-off — a daily or hourly run is what turns this into a monitoring system.

Run it on a schedule

A one-off pull answers a question; a schedule answers it every day without you. Open Schedules in the Apify Console, point a cron at this Actor, and the dataset keeps filling on its own — no server, no cron box, no babysitting. Everything here is built to be re-run: you are billed per article delivered, so a scheduled run that finds nothing new costs nothing.

FAQ

Do I get the real article URL or a Google redirect link?

The real publisher URL, resolved for you. Redirect links are useless downstream, which is the single most common complaint about news scrapers.

Can I get the full article text, not just the headline?

Yes — full text, author and lead image where the publisher exposes them.

How far back can I go?

Far past the ~100-item ceiling a single query returns: the archive Actor in this suite walks the date range window by window.

Can I filter by topic or country?

Yes. Query, topic feed, language and country are all inputs, and can be combined in one run.

---

Something wrong, or a field you need that is missing? Open an issue on the Issues tab — it is read and it gets fixed. If this saved you time, a rating on the Store page helps the next person find it.