Google News Scraper — Real Publisher URLs, Agent-Ready

Scrape Google News by search query, topic section, or top headlines and get back clean, flat JSON — with the one field most Google News scrapers get wrong: the real publisher article URL, not Google's obfuscated redirect.

Why this scraper

Google News RSS gives you links like https://news.google.com/rss/articles/CBMiigF... — an opaque redirect, not the article. Google changed how these are encoded in 2024, which broke a lot of scrapers (you get a Google URL that a browser can open but your code/database/LLM can't use). This actor resolves each link to the actual publisher URL (e.g. https://www.theguardian.com/...), so the data is usable the moment it lands.

• ✅ Resolved publisher URLs — the headline feature. Falls back gracefully to the Google URL with url_resolved: false if a specific link can't be resolved, so a run never fails on one bad article.
• ✅ Search, topics, and headlines — free-text queries (with Google News operators like when:1d, site:, quotes), 8 topic sections, and the top-headlines feed.
• ✅ Any language / country edition — hl + gl (e.g. pt-BR / BR, es-419 / MX).
• ✅ Flat schema for LLMs & pipelines — one row per article, typed fields, absolute URLs, explicit url_resolved flag. No nested junk, no formatted-string duplicates.
• ✅ Fast & cheap — HTTP-only, no headless browser. Runs at 512 MB.

Input

Field	Type	Description
queries	string[]	Free-text Google News searches. Operators supported ("exact phrase", when:7d, site:bbc.com).
topics	string[]	Any of WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH.
headlines	boolean	Include the top-headlines feed.
language	string	UI language hl, e.g. en-US, pt-BR. Default en-US.
country	string	Country edition gl, e.g. US, BR, GB. Default US.
maxArticlesPerQuery	integer	1–100 per feed (RSS returns ~100 max). Default 100.
resolvePublisherUrls	boolean	Resolve real publisher URLs. Default true.
dedupByUrl	boolean	Drop duplicates across feeds. Default true.

Example input

{
  "queries": ["\"artificial intelligence\"", "nvidia when:1d"],
  "topics": ["TECHNOLOGY"],
  "headlines": false,
  "language": "en-US",
  "country": "US",
  "maxArticlesPerQuery": 50,
  "resolvePublisherUrls": true
}

Output

One dataset item per article:

{
  "query": "artificial intelligence",
  "query_type": "search",
  "language": "en-US",
  "country": "US",
  "title": "Apple sues OpenAI, alleging AI company stole trade secrets",
  "publisher": "The Guardian",
  "publisher_domain": "www.theguardian.com",
  "article_url": "https://www.theguardian.com/technology/2026/jul/10/apple-sues-openai-trade-secrets",
  "google_url": "https://news.google.com/rss/articles/CBMiigF...",
  "url_resolved": true,
  "published_at": "2026-07-10T22:33:00.000Z",
  "scraped_at": "2026-07-12T18:00:00.000Z"
}

Use cases

• News monitoring & alerts — track a brand, ticker, or topic across publishers.
• Media research & datasets — build clean article-URL datasets for analysis or ML.
• LLM / RAG pipelines — feed resolved URLs straight into a fetch-and-summarize step.
• Market & competitor intelligence — topic and query feeds per country edition.

Notes & limits

• Article full text is not extracted — you get the resolved URL; fetch the body downstream if you need it.
• Google News RSS returns up to ~100 items per feed; use multiple queries for breadth.
• Reliability is monitored continuously against a golden probe set; the target is < 2% run failure over any 30-day window.