Google News Scraper

Scrapes Google News RSS feeds — search terms, topic sections and top stories — and, unlike most scrapers in this category, gives you the publisher's real article URL, not Google's redirect link.

The redirect problem, and what this actor actually does about it

Every link in a Google News feed looks like this:

https://news.google.com/rss/articles/CBMilgFBVV95cUxQTU9VbHZPQ2czTUlfNVZMVk53...?oc=5

That is not the article. It is a Google redirect, and it is the single most common complaint about scrapers in this category: they hand you that string and call it the article URL.

Two things were measured on 2026-08-24 and are worth stating plainly:

• The identifier does not decode to a URL. In feeds published before 2024, the CBMi... segment was base64 of a tiny protobuf whose only string was the destination, and decoding it was enough. Current feeds carry an opaque AU_yqL... token instead. This actor still decodes the old format when it meets one, because it is free — but on live feeds today it never fires.
• Following redirects does not work either. The link 302s to a Google News interstitial that performs the real navigation from JavaScript. A plain HTTP client sees a 592 KB app shell whose canonical URL points back at news.google.com.

What does work, with no browser and no headless Chrome: fetch the interstitial, read the per-article signature it embeds, and post it to Google News's own batchexecute endpoint — the same request the page's own JavaScript makes. In testing this resolved 25 out of 25 articles.

So the output carries both fields, and they are never confused:

field	what it is
google_news_url	Google's redirect link, always present
resolved_url	the publisher's own URL, or null

resolved_url is null when — and only when — resolution genuinely failed (Google refused the interstitial, or the signature endpoint changed). It is never quietly filled with the redirect URL. Set resolveUrls to false if you do not need it: resolution costs one extra page fetch per article, so turning it off makes runs noticeably faster and cheaper.

Limits you should know before paying for a run

• Google caps a search feed at about 100 articles per query, and there is no pagination. Measured on 2026-08-24: a search feed returned 100 items, a topic feed 70, and the top-stories feed 38. No parameter moves those numbers. To go past 100 you need more queries — narrower terms, or the same term split across timeRange windows — not a bigger maxItems.
• timeRange applies to search queries only. Google implements it by appending a when: token to the query text, so topic: and top feeds ignore it.
• The feed is a headline feed, not an article feed. See the next section.

Output fields

field	notes
guid	Google's stable article identifier; runs are de-duplicated on it
title	headline, with the source name appended by Google
google_news_url	the redirect link, always present
resolved_url	publisher URL, or null — see above
published_at	RFC 822 timestamp exactly as the feed gives it
source_name	publication name, e.g. The Guardian
source_url	publication homepage, e.g. https://www.theguardian.com
description_html	Google's snippet markup
description_text	the same snippet with tags stripped
query	which input entry produced this article
language, country	the edition that was read

What the source does not give, and is therefore not invented here: article body text, author, image, word count, sentiment, and any engagement or share count. Google News RSS carries none of them. description is an anchor tag plus the source name — a link, not a summary — so description_text is short by nature. Anything absent is null, never 0 or "", because a zero reads as a measurement.

Input

• queries — mix freely: openai (search), topic:TECHNOLOGY (section), top (top stories). Topics are WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH.
• language / country — Google's hl and gl. The ceid parameter is built from both automatically (hl=en-US + gl=US gives ceid=US:en).
• timeRange — 1h, 1d, 7d or 1y, search queries only.
• maxItems — total across all queries.
• resolveUrls — on by default.
• proxyConfiguration — recommended. Google rate-limits per address, and the actor rotates its exit address between queries and on every retry.

Reliability

Refusals (HTTP 403, 429, 5xx, and Google's unusual-traffic page) are treated as transient and retried from a fresh proxy address. A malformed feed, an unknown topic section, or a query that simply matched nothing is treated as permanent and is not retried — retrying those would spend your money to learn the same answer four times. A query that returns nothing is reported in the log and is not charged for.