Google News Scraper

Scrape search results, topic sections, local news and top headlines from Google News — in any language and edition, in one run.

What it covers

Most Google News Actors do search only. This one fetches every feed type Google exposes:

• Search — with Google News operators (tesla source:reuters, "climate policy") and a recency filter (1h, 1d, 7d).
• Topics — WORLD, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH, or a topic id lifted from a Google News URL.
• Places — local news for a named city or region.
• Top headlines — the front page of the chosen edition.
• Any edition — combine language and country (de/DE, es-419/MX, en-GB/GB) to read the news as that audience sees it.

Every article carries its publisher, publication timestamp, snippet, and which feed produced it. Duplicate stories across feeds are removed by default.

Input

Field	Type	Description
queries	array	Search terms; Google News operators supported
topics	array	Section names or topic ids
places	array	Local news by place name
includeTopHeadlines	boolean	Add the edition's front page
language / country	string	Edition, e.g. en-US / US
timeFilter	string	1h, 12h, 1d, 7d — search only
maxArticlesPerFeed	integer	Up to 100 (Google's own ceiling)
maxItems	integer	Total cap (0 = no limit)
deduplicate	boolean	Drop stories already seen in another feed
concurrency / requestDelayMs	integer	Throughput controls
proxyConfiguration	object	Optional — not required

Example

{
  "queries": ["openai", "tesla"],
  "topics": ["TECHNOLOGY"],
  "places": ["London"],
  "includeTopHeadlines": true,
  "language": "en-US",
  "country": "US",
  "timeFilter": "7d"
}

Output

{
  "title": "The first anti-AI protester to be jailed has a message for OpenAI…",
  "url": "https://news.google.com/rss/articles/CBMiWkFVX3lxTFBZ…",
  "publisher": "The Guardian",
  "publishedAt": "2026-08-13T17:02:46.000Z",
  "snippet": "The first anti-AI protester to be jailed has a message…",
  "source": "search:openai",
  "sourceType": "search",
  "language": "en-US",
  "country": "US"
}

A feed that fails produces an item with an error field rather than disappearing.

Notes and limits

• Article links are Google News redirect URLs. Google no longer exposes the publisher's direct URL in its feed — the link resolves to the article in a browser, but it is not a clean publisher link. The publisher field gives you the outlet name, which is what most pipelines actually need. Any tool claiming otherwise is either following each link (slow and costly) or returning stale data.
• Google caps each feed at roughly 100 articles. Ask for more and you get what exists. To widen coverage, use more queries, topics or editions rather than a bigger limit.
• Snippets are Google's own, not full article text — usually the headline plus the outlet.
• Editions matter. The same query in en-US/US and de/DE returns genuinely different coverage, not translations.

Local development

npm install
npm run build
node dist/main.js     # reads storage/key_value_stores/default/INPUT.json