Google News Scraper: Articles, Publisher URLs & Full Text

Google News Scraper collects news articles from Google News: search results (with all the Google News operators), topic sections and top stories, in 75 country/language editions. Every article comes with its headline, publisher, publication time and the real publisher URL, not the news.google.com/rss/articles/... redirect link that Google's feed gives you.

What makes it different from other Google News scrapers:

• 🔗 Real publisher URLs, included in the price. Google News links are encrypted redirects. This Actor resolves them to the actual article URL (e.g. https://www.cnbc.com/2026/11/02/...html) at no extra charge. Other scrapers either give you the Google redirect or charge a separate event for each resolved URL.
• 📅 More than 100 results per query. Google News returns at most ~100 articles per search. Set a date range and the Actor runs one search per day (date slicing), then merges the days and removes duplicates. In our test, one query over 3 days returned 263 unique articles instead of 100.
• 📰 Optional full article text. Turn on full text to get the main text, author, published time, main image and description from the publisher page. You pay for text only when it was actually extracted: paywalled and blocked pages are flagged and cost nothing extra.
• 🔔 Monitoring mode. "Only new articles" remembers what earlier runs returned, per query and edition, so a scheduled run gives you just the new articles. It works well for brand alerts, competitor tracking and news feeds.

No login, no browser, no proxy needed by default. You pay per article.

What data can you get?

Field	Example	Notes
title	Defense, healthcare, crypto and AI: How the midterm election could affect stocks	Without the " - Publisher" suffix
publisher, publisherHomepage, publisherDomain	CNBC, https://www.cnbc.com, cnbc.com	From Google News
publisherUrl	https://www.cnbc.com/2026/11/02/midterm-election-stocks-defense-healthcare-crypto-ai.html	Real article URL (included in the price)
publisherUrlResolved	true	false if Google refused to resolve the link (publisherUrl is then the Google link)
googleNewsUrl, googleNewsId	https://news.google.com/rss/articles/CBMi...	The original Google News link
publishedAt	2026-11-02T11:00:01.000Z	ISO 8601, UTC
query / topic / sourceType	artificial intelligence / null / search	What returned the article
rank	3	Position in the Google News feed
relatedArticles, relatedArticlesCount	[{ title, publisher, googleNewsUrl }, ...]	Other coverage Google groups with the story (topic feeds and top stories)
searchDay	2026-10-31	With date slicing: the day searched
edition, language, country	US:en, en-US, US	Google News edition
snippet	null	Google News RSS no longer includes summaries; filled only when the feed has text beyond the headline
Full text mode: text, wordCount	"Lockheed Martin gets about 70% of its sales..." , 1547	Main article text, paragraphs separated by blank lines
author, articlePublishedAt, imageUrl, description	Luke Fountain, 2026-11-02T11:00:01Z, https://image.cnbcfm.com/..., "Democratic gains in Congress..."	From the publisher page (meta tags and JSON-LD)
textExtracted, textError, paywalled	true, null, false	Why text is missing: paywall, blocked (HTTP 403), ...
id, scrapedAt	b2ecca80b781999a2e1f	Stable ID (hash of the publisher URL)

Use cases

• Media monitoring and PR: track every mention of your brand, CEO or product, scheduled hourly, and receive only new articles.
• Competitor and market intelligence: follow competitors, deals and regulation with site:, intitle: and exact-phrase queries.
• Financial and trading signals: news flow per ticker or company with exact timestamps and publisher URLs.
• AI and LLM pipelines: build RAG corpora and news summaries, and train classifiers on full article text with real source URLs.
• Research and journalism: collect a month of coverage on a topic, day by day, across countries and languages.
• News apps and newsletters: current top stories and section feeds in 75 editions, with related coverage per story.

How to use it

1. Enter one or more search queries, and/or pick topic feeds (Top stories, World, Business, Technology...).
2. Choose the edition (country and language), e.g. United States (English), United Kingdom, Germany, India, Brazil.
3. Set Max articles per query, then click Start.

Google News search operators

All of them work inside a query:

Operator	Example	Meaning
"..."	"electric vehicles"	Exact phrase
site:	tesla site:reuters.com	Only one publisher
intitle:	intitle:nvidia	Word in the headline
- / OR	apple -fruit, openai OR anthropic	Exclude / either term
when:	bitcoin when:7d, when:12h	Last N hours/days/months
after: / before:	after:2026-10-01 before:2026-10-15	Date range (US Pacific days; before: is exclusive)

Examples

Latest articles for a query (the prefilled input):

{
    "queries": ["artificial intelligence"],
    "edition": "US:en",
    "maxArticlesPerQuery": 10,
    "decodeUrls": true,
    "extractArticleText": false
}

A whole month of coverage, past the 100-result cap (date slicing):

{
    "queries": ["\"OpenAI\""],
    "dateFrom": "30 days",
    "splitByDay": true,
    "maxArticlesPerQuery": 0
}

The Actor runs 31 searches (one per day, newest first), merges them and removes duplicates. A query with when:30d or after:/before: in it is sliced the same way.

Top stories and sections in the UK edition, with full text:

{
    "topics": ["TOP_STORIES", "BUSINESS", "TECHNOLOGY"],
    "edition": "GB:en",
    "maxArticlesPerQuery": 30,
    "extractArticleText": true
}

Hourly brand monitor (only new articles):

{
    "queries": ["\"Acme Corp\"", "\"Acme\" intitle:recall"],
    "edition": "US:en",
    "onlyNewArticles": true,
    "stateKey": "acme-monitor"
}

Schedule it in Schedules (e.g. every hour) and connect a webhook, Slack, email or Zapier to the run's dataset. The first run returns the current articles and saves them as the baseline. Each later run returns only articles you haven't seen yet, and you pay only for those.

Output example

Real output from a local run (samples/sample-output.json, text shortened):

{
    "id": "b2ecca80b781999a2e1f",
    "title": "Defense, healthcare, crypto and AI: How the midterm election could affect stocks",
    "publisher": "CNBC",
    "publisherHomepage": "https://www.cnbc.com",
    "publisherDomain": "cnbc.com",
    "publisherUrl": "https://www.cnbc.com/2026/11/02/midterm-election-stocks-defense-healthcare-crypto-ai.html",
    "publisherUrlResolved": true,
    "publisherUrlError": null,
    "url": "https://www.cnbc.com/2026/11/02/midterm-election-stocks-defense-healthcare-crypto-ai.html",
    "googleNewsUrl": "https://news.google.com/rss/articles/CBMikwFBVV95cUxNNTJxMVlLZVJ1YmQwOGw0Y2R0b3JqQS04aWZz...?oc=5",
    "googleNewsId": "CBMikwFBVV95cUxNNTJxMVlLZVJ1YmQwOGw0Y2R0b3JqQS04aWZz...",
    "publishedAt": "2026-11-02T11:00:01.000Z",
    "snippet": null,
    "relatedArticles": [],
    "relatedArticlesCount": 0,
    "sourceType": "search",
    "query": "artificial intelligence",
    "topic": null,
    "searchDay": null,
    "rank": 3,
    "edition": "US:en",
    "language": "en-US",
    "country": "US",
    "textExtracted": true,
    "textError": null,
    "text": "Lockheed Martin gets about 70% of its sales from the U.S. government. Medicaid accounts for three-quarters of Molina Healthcare's premium revenue ...",
    "wordCount": 1547,
    "author": "Luke Fountain",
    "description": "Democratic gains in Congress could bring tougher oversight of major companies and shift billions of dollars in federal spending and regulatory policy.",
    "imageUrl": "https://image.cnbcfm.com/api/v1/image/106949322-1632916362498-gettyimages-1235565564-848033_NA-POL-0928-CAPITOL-HILL_KKN_29317JPG.jpg?v=1793390260&w=1920&h=1080",
    "articlePublishedAt": "2026-11-02T11:00:01.000Z",
    "articleLanguage": "en",
    "paywalled": false,
    "scrapedAt": "2026-11-02T13:28:51.246Z"
}

A RUN_SUMMARY record in the run's key-value store lists every query and feed with its status, counts, duplicates, URL-decoding and full-text statistics, and request counts.

Pricing

Pay per event, no start fee:

Event	Price	When
article	$0.001	Every article in the output, including its resolved publisher URL
article-text	$0.002	Only when full text was requested and successfully extracted

Worked examples:

• 1,000 articles with publisher URLs ≈ $1.00.
• 1,000 articles with full text, if 650 texts are extracted ≈ $1.00 + $1.30 = $2.30. Paywalled or blocked pages cost $0.001 (the article only).
• An hourly monitor finding 20 new articles per run ≈ $0.02 per run, about $14 a month.

Set Maximum cost per run on the run to cap your spend: the Actor stops cleanly when the budget is reached.

Tips and limits

• ~100 articles per search request is a Google limit. Use a date range with date slicing to go beyond it. If a single day still returns ~100 results (the run log says so), narrow the query (e.g. add site: or exact phrases) to get complete coverage of that day.
• Date slicing costs one Google request per day searched, and stops as soon as "Max articles per query" is reached, so raise that limit (or set 0) for full ranges. Days are US Pacific calendar days, as Google defines them.
• Topic feeds and top stories return Google's current selection (~40-70 articles per section). The date range applies to search queries only.
• Resolving publisher URLs takes about one request to Google per article (plus one per 10 articles). The Actor runs 2 in parallel and backs off automatically if Google rate-limits. If Google refuses, the article still comes out, with the Google link and publisherUrlResolved: false. A run never fails because of URL decoding. Resolved URLs are remembered (google-news-url-cache store), so repeat runs resolve known articles instantly. Turn decoding off for the fastest possible run.
• Full text is best-effort. It works on most open news sites. Hard paywalls (in our tests the Washington Post; the New York Times answered HTTP 403), registration walls and bot-protected sites return text: null with a textError reason, and you are not charged for their text. Metered sites that serve the full article are extracted, with paywalled: true. Each site gets at most one request per second.
• Snippets: Google News RSS no longer includes article summaries (the description repeats the headline), so snippet is usually null. Turn on full text to get the publisher's description and the article text.
• Related coverage links are Google News links (not resolved, to keep runs fast and cheap).
• Memory: 256 MB is enough for most runs. Use 512 MB for large full-text runs.
• Proxy: not needed normally. If a run reports HTTP 429/403 from Google, enable Apify Proxy (a US location is best: EU IPs can get Google's cookie-consent page).

FAQ

Is it legal to scrape Google News? This Actor reads Google News' public RSS feeds and public article pages, without logging in. It does not collect personal data beyond what publishers print publicly (such as an article's author byline). You are responsible for how you use the data: respect copyright (store and republish article text only where you have the right to), the publishers' terms, and laws such as GDPR. If in doubt, ask a lawyer.

Why do other scrapers give me news.google.com links? Since 2024 Google News encrypts article links. Resolving them needs extra requests to Google, so many scrapers skip it or charge extra. Here it is included in the article price.

How many articles can I get for one query? Without a date range: up to ~100 (Google's limit). With a date range and date slicing: up to ~100 per day, e.g. thousands for a month.

How does "only new articles" work? The Actor stores the IDs of the articles it has output in a named key-value store (google-news-scraper-state), separately for each query/topic and edition, under your State key. Later runs with the same state key skip those articles. Use a different state key for each independent monitor.

Can I get news in other languages and countries? Yes, pick one of 75 editions (e.g. Germany (Deutsch), France, Japan, Brazil, India (हिन्दी), Mexico). Run the Actor once per edition if you need several.

Why is publisherUrlResolved false for some articles? Google sometimes refuses or rate-limits the decoding request. The article is still output with its Google News link. publisherUrlError tells you why.

Does it use a browser? No. It uses fast HTTP requests only, so it is cheap and quick.

Integrations

• API: run the Actor and fetch the dataset via the Apify API or the JavaScript/Python clients.
• Make, Zapier, n8n: use the Apify app/node to start runs and read the dataset, e.g. post every new article to Slack, Google Sheets or a CRM.
• Webhooks: trigger your own endpoint when a run finishes.
• MCP / AI agents: use the Actor as a tool through the Apify MCP server so assistants such as Claude can search Google News and read articles.
• Export to JSON, CSV, Excel, XML or RSS from the dataset tab.

Changelog

• 0.1 (2026-11): First release: Google News search (all operators), topic feeds and top stories in 75 editions; publisher URL decoding included (batched, cached, with graceful fallback); day-by-day date slicing past the 100-result cap; optional full text (Readability) with paywall/block detection; incremental "only new articles" mode per query and edition.

More scrapers by the same developer

• Substack Scraper
• Google Ads Transparency Scraper
• Telegram Channel Scraper
• ATS Jobs Scraper: Greenhouse, Lever, Ashby & Workday
• Dice Jobs Scraper
• SEEK Jobs Scraper

Browse all my Actors on my Apify Store profile.