Google News Scraper

Scrape Google News at scale and get back the real publisher URL for every article — not the opaque news.google.com/rss/articles/CBMi... redirect that most Google News scrapers leave you with. Search by keyword, browse a topic section, filter by publisher or date range, and run the same query across 50+ country editions and 40 languages in a single run.

No API key. No login. No browser. Typically under 15 seconds for 100 fully-resolved articles.

What this Google News scraper does

This Actor turns Google News into a clean, structured dataset you can query, diff and monitor. Point it at a keyword and it returns every matching article Google is currently surfacing, with the headline, publisher, publication timestamp, direct article link, and Google's own story-clustering (which other outlets are covering the same story).

Why "resolve the real URL" matters

Every link in a Google News feed is an encoded redirect. Since Google retired the old encoding scheme, those identifiers no longer contain the destination — they are opaque protobuf handles. A scraper that simply returns them hands you data you cannot use: you cannot deduplicate by domain, you cannot tell a wire reprint from an original, you cannot feed the URL to a summariser, and the link rots.

This Actor calls Google's own internal resolution endpoint to recover the true publisher URL for each article — in batches of 20 per request, so resolution costs about one extra second per 20 articles rather than one page load each. In testing across US, UK, German and Japanese editions, resolution succeeded for 100% of articles.

How it works

1. Build the feed URLs. Your input is expanded into one RSS feed per (query × market) combination. Topic sections use a generated Google News topic token, so built-in sections like Business or Technology work in every locale — not just US English.
2. Fetch feeds concurrently. Each feed returns up to ~100 articles. Results are merged and de-duplicated by Google's article ID.
3. Resolve publisher URLs. Article signatures are harvested in parallel, then decoded in batched RPC calls.
4. Optionally fetch thumbnails from each publisher's own og:image tag.
5. Push to the dataset, honouring your run's charge limit.

The pipeline is built to degrade rather than fail. If URL resolution or image lookup times out, articles are still returned with their Google News link intact, and the run succeeds. A per-run enrichment time budget guarantees the Actor never runs long.

Input

Field	Type	Default	Description
query	string	—	Keyword or phrase. Supports Google News operators: OR, -exclude, "exact phrase", intitle:, allintext:
topic	enum	—	Built-in section: World, Nation, Business, Technology, Entertainment, Sports, Science, Health
publisher	string	—	Restrict to one domain, e.g. reuters.com
language	enum	en	40 languages
region	enum	US	56 country editions
additionalMarkets	array	[]	Extra editions as REGION:language, e.g. ["GB:en","DE:de","JP:ja"]
dateFrom / dateTo	date	—	Publication window (YYYY-MM-DD)
maxItems	integer	50	Total articles across all feeds
resolveArticleUrls	boolean	true	Recover real publisher URLs
includeImages	boolean	false	Fetch article thumbnails

Leave every field empty and the Actor scrapes the Top Stories front page for the selected market, so a bare run always returns data.

Example input

{
  "query": "semiconductor export controls",
  "additionalMarkets": ["GB:en", "DE:de", "JP:ja"],
  "dateFrom": "2026-08-01",
  "maxItems": 200,
  "resolveArticleUrls": true
}

Output example

One dataset item per article. Real output from a TECHNOLOGY topic run:

{
  "position": 3,
  "title": "Five ways that the new Siri AI is frustrating iPhone and Apple Watch users",
  "description": "Five ways that the new Siri AI is frustrating iPhone and Apple Watch users",
  "url": "https://9to5mac.com/2026/09/20/five-ways-that-the-new-siri-ai-is-frustrating-iphone-and-apple-watch-users/",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMiqgFBVV95cUxNZUV2WFp0U0x5Tnl3cTJCQ3Q1N2htcXlxT0lvXzUzbmhsSHJoZEU3MHJ4V2Fy...?oc=5",
  "urlResolved": true,
  "source": "9to5Mac",
  "sourceUrl": "https://9to5mac.com",
  "sourceDomain": "9to5mac.com",
  "image": "https://9to5mac.com/wp-content/uploads/sites/6/2026/09/iphone-18-pro-ios-27-siri-ai.jpg",
  "publishedAt": "2026-09-20T18:35:00Z",
  "publishedAtRaw": "Sun, 20 Sep 2026 18:35:00 GMT",
  "relatedCoverage": [
    { "title": "The Top New Features in Apple's iOS 27 and iPadOS 27", "source": "WIRED", "googleNewsUrl": "https://news.google.com/rss/articles/CBMinwFBVV95cUxOald3VG43..." },
    { "title": "With iOS 27, Apple Is Laying the Groundwork to Paywall Your iPhone", "source": "pcmag.com", "googleNewsUrl": "https://news.google.com/rss/articles/CBMinAFBVV95cUxOeGUxR09n..." }
  ],
  "relatedCoverageCount": 4,
  "query": null,
  "topic": "TECHNOLOGY",
  "language": "en",
  "region": "US",
  "market": "US:en",
  "articleId": "CBMiqgFBVV95cUxNZUV2WFp0U0x5Tnl3cTJCQ3Q1N2htcXlxT0lvXzUz...",
  "scrapedAt": "2026-09-20T23:37:14Z"
}

relatedCoverage is Google's own story clustering — the other outlets running the same story. On topic and Top Stories feeds it is populated for the large majority of items, which makes it useful for measuring share of voice and spotting wire pickup.

Pricing

This Actor uses pay per result. You are charged per article written to the dataset, and nothing else — no platform usage charges, no per-run fee beyond the standard Actor start event. A run that returns 40 articles costs 40 results.

Because articles are de-duplicated across markets before pushing, running one query against five country editions does not bill you five times for the same syndicated story.

Use cases

Media monitoring and PR. Track mentions of a brand, executive or product across countries, and see which outlets picked up a story first via relatedCoverage.

Competitive and market intelligence. Watch competitor announcements, funding, regulatory action or recalls. The publisher filter plus a date window turns this into a targeted archive query.

Building a news dataset for LLMs and RAG. Resolved publisher URLs are directly fetchable, which is exactly what a downstream summarisation, embedding or classification pipeline needs. Google redirect links are not.

Finance and event-driven research. Schedule a run every 15 minutes on a ticker or commodity and diff the dataset to get a clean stream of new headlines with reliable timestamps.

Google News API alternative. Google has no official public News API. This Actor is a stable, structured, no-authentication substitute you can call from the Apify API, CLI, MCP, or any scheduler.

Cross-market narrative comparison. Run the same query in US:en, DE:de and JP:ja and compare which outlets and framings dominate each edition.

FAQ & Support

Do I need a Google API key or account? No. The Actor uses no authentication of any kind.

How many articles can I get per query? Google News serves a maximum of roughly 100 items per feed, and that is a hard limit on Google's side — no scraper can page beyond it. To go further, widen the search: add markets via additionalMarkets, split a broad topic into several narrower queries, or slice a date range into windows. Each feed contributes up to ~100 more articles.

How far back does the date filter reach? Google honours dateFrom/dateTo for roughly the last 12 months on most queries. The filter is Google's own and is approximate at the edges — expect a small number of results just outside the window.

Why is description often the same as title? For single-article items Google News RSS does not provide a separate snippet; the feed's description field contains only the headline and source. This Actor reports that faithfully rather than inventing a summary. Cluster items carry real extra information in relatedCoverage.

Why do some articles have no image? Thumbnails are read from the publisher's own page, and some publishers block non-browser requests. Expect a thumbnail for roughly two thirds of articles when includeImages is on. Image lookup is off by default because it materially increases run time.

Can it scrape the full article text? No, by design. This Actor returns Google News metadata and the canonical article URL. Full-text extraction is a separate concern with its own copyright considerations — pipe the url field into a dedicated article-extraction Actor if you need body text.

Does it need a proxy? No. Google News RSS is reachable from Apify's datacenter IPs. Apify Proxy is available in the input for very large jobs but is off by default, and a proxy failure degrades to a direct connection rather than failing the run.

Is scraping Google News legal? This Actor reads only publicly available, non-personal Google News RSS feeds — no login, no personal data, no paywalled content. You are responsible for how you use the output, particularly in respect of publishers' copyright in headlines and article text.

Something looks wrong. Open an issue on the Actor's Issues tab with your input JSON and the run ID. Google occasionally changes the article-ID encoding; if resolution success drops, that is the cause and it will be fixed promptly.