Google News Scraper with Publisher URLs

Get Google News articles for any search or topic with the real publisher URL instead of a news.google.com redirect, for $0.0015 per article plus $0.0005 when the publisher URL is found.

Google News feeds only give news.google.com/rss/articles/... links. This Actor turns each one into the publisher's own URL (for example https://www.theguardian.com/technology/2026/nov/03/google-search-artificial-intelligence), so you can open, store or crawl the article directly.

• Search with full Google News syntax: "exact phrase", OR, -exclude, site:reuters.com, intitle:, when:7d.
• Read topic feeds. Top stories, World, Nation, Business, Technology, Entertainment, Science, Sports, Health, or any topic ID or main topic URL.
• Pick from 85 editions (country and language), from US:en and GB:en to DE:de, BR:pt-419, JP:ja and IN:hi.
• Filter by date with from/to dates or relative ones ("24 hours", "7 days"), checked against each article's exact publication time.
• Get past Google's 100-result cap. Above 100 articles, each search is read one day at a time, and duplicates across queries, topics and editions are removed before charging.

Press Start with the empty form and you get 25 articles about "artificial intelligence" from the US edition, with publisher URLs, for $0.05.

What you get

One dataset row per article.

Field	Example	Notes
title	Google says there’s no way to use Search without using artificial intelligence	Without the " - Publisher" suffix Google adds
source	The Guardian	Publisher name
publishedAt	2026-11-03T21:47:00.000Z	ISO 8601, UTC
publisherUrl	https://www.theguardian.com/technology/...	The real article URL. null if resolution is off or failed
publisherDomain	theguardian.com	Host of publisherUrl (or of sourceUrl when publisherUrl is null)
sourceUrl	https://www.theguardian.com	Publisher home page, from the feed
googleNewsUrl	https://news.google.com/rss/articles/CBMi...?oc=5	Original Google News link
articleId	CBMi...	Google News article ID
resolveStatus	resolved	resolved, failed, skipped (not tried, see below) or off
resolveError	null	Why publisherUrl is null
relatedArticles	[{ "title", "source", "googleNewsUrl" }]	Other coverage of the same story (top-stories clusters). Not resolved
relatedCount	4	Number of related articles
feedType, query, topic	search, tesla, null	Which input line found the article
edition, language, country	US:en, en-US, US	Edition the article came from
rank	1	Position in Google's feed
scrapedAt	2026-11-05T12:55:05.955Z	When the row was saved

Google News feeds have no article text, snippet, image or author. You get the title, publisher, date and link. To get the article text, send publisherUrl to a crawler, for example our Web Fetch to Markdown Actor.

The dataset has two views. Articles is the overview table, and Links shows the publisher URL next to the Google link and the resolution result.

Example output

This row comes from a run on Apify on 2026-11-05 with the empty form (search "artificial intelligence", edition US:en). All 25 articles of that run got a publisher URL.

{
  "title": "Commission turns to artificial intelligence to write better laws",
  "source": "POLITICO.eu",
  "publishedAt": "2026-11-04T12:04:00.000Z",
  "publisherUrl": "https://www.politico.eu/article/commission-turns-to-artificial-intelligence-to-write-better-laws/",
  "publisherDomain": "politico.eu",
  "sourceUrl": "https://www.politico.eu",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMingFBVV95cUxQdHBzLUluelV6TG1RVkNCVWFjb0lwX0I5aXdNM195SkI5TUVuRXV6cUVlVWNxeXgwN1pBUTJoZ3dQdldXVHE3Q2JtNl90WmVmUUtMX0ltTUQzQjZoVXhFc1l6UmJia0pwdmxVZjAxMnYyN05iWHpCeUJQSlgyWTB2bFFOUmZPaVRWZWVRcVdIbGUycFRTa0dySkctVnNvQQ?oc=5",
  "articleId": "CBMingFBVV95cUxQdHBzLUluelV6TG1RVkNCVWFjb0lwX0I5aXdNM195SkI5TUVuRXV6cUVlVWNxeXgwN1pBUTJoZ3dQdldXVHE3Q2JtNl90WmVmUUtMX0ltTUQzQjZoVXhFc1l6UmJia0pwdmxVZjAxMnYyN05iWHpCeUJQSlgyWTB2bFFOUmZPaVRWZWVRcVdIbGUycFRTa0dySkctVnNvQQ",
  "resolveStatus": "resolved",
  "resolveError": null,
  "relatedArticles": [],
  "relatedCount": 0,
  "feedType": "search",
  "query": "artificial intelligence",
  "topic": null,
  "edition": "US:en",
  "language": "en-US",
  "country": "US",
  "rank": 1,
  "scrapedAt": "2026-11-05T14:10:47.420Z"
}

Pricing

Pay per event. You only pay for articles that end up in your dataset.

Event	Price	When
article	$0.0015	Each article row saved (after date filters and duplicate removal)
url-resolved	$0.0005	Each saved article whose publisher URL was found

Some worked examples.

Run	Articles	URLs found	Cost
Empty form (one click)	25	25	25 × $0.002 = $0.05
1,000 articles with publisher URLs	1,000	1,000	$1.50 + $0.50 = $2.00
1,000 articles, "Resolve publisher URLs" off	1,000	0	$1.50
10 queries × 3 editions × 25 articles, URLs on	750	750	750 × $0.002 = $1.50

The last row assumes no duplicates. Duplicates are removed first and aren't charged, so the real cost can be lower. Failed or skipped resolutions and failed feeds aren't charged either.

You can set a maximum cost per run in the run options. The Actor stops before it resolves or saves rows it can't pay for.

Input

Field	Default	What it does
Search queries	none	One search per line, up to 100. If both queries and topics are empty, the Actor runs the example search artificial intelligence.
Topic feeds	none	Up to 50: TOP, WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SCIENCE, SPORTS, HEALTH, a topic ID (CAAq...) or a news.google.com/topics/... URL. Subsection URLs (with /sections/ in them) aren't supported. They're skipped with a warning, and the rest of the input runs.
Editions	US:en	Up to 20 editions. Every query and topic runs in each edition (at most 500 feeds per run).
Max articles per query or topic	25	1 to 5,000, per query or topic and per edition.
From date / To date	none	YYYY-MM-DD (whole UTC days) or relative: 24 hours, 7 days, 2 weeks, 3 months, 1 year (at most 10 years back).
Resolve publisher URLs	on	Fill publisherUrl. Turn it off if the Google link is enough (cheaper and faster).
Remove duplicates	on	Skip an article already saved in this run: same Google ID, same publisher URL, or same title and publisher published less than 12 hours apart. A daily column that reuses its headline ("Mortgage rates today") is kept once per day.
Parallel URL resolutions	3	1 to 10.
Proxy	none	Optional Apify Proxy (datacenter). See "Blocks and proxies".

This input gets Tesla and Rivian news from the last 7 days in the US and UK editions, plus UK business headlines.

{
  "queries": ["tesla", "rivian -stock"],
  "topics": ["BUSINESS"],
  "editions": ["US:en", "GB:en"],
  "dateFrom": "7 days",
  "maxArticlesPerQuery": 100
}

Use it from the API

The synchronous endpoint runs the Actor and returns the dataset rows in one HTTP call. The call waits at most 300 seconds for the run. If the run takes longer, the call ends without the rows, but the run keeps going. So use it for small runs and start a normal run for big ones.

curl -s -X POST \
  "https://api.apify.com/v2/acts/zovo-one~google-news-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["openai"], "dateFrom": "24 hours", "maxArticlesPerQuery": 20}'

Python

Install the client with pip install apify-client.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("zovo-one/google-news-scraper").call(run_input={
    "queries": ["tesla", "rivian -stock"],
    "editions": ["US:en", "GB:en"],
    "dateFrom": "7 days",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["publishedAt"], item["source"], item["publisherUrl"])

Node.js

Install the client with npm install apify-client.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('zovo-one/google-news-scraper').call({
    topics: ['TECHNOLOGY'],
    editions: ['US:en'],
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) console.log(a.publishedAt, a.source, a.publisherUrl);

Use it from AI agents (MCP)

The Apify MCP server at mcp.apify.com can add this Actor as a tool for AI agents. Pin it with the tools parameter. Each MCP client sets up a remote server in its own way, and the configurator at https://mcp.apify.com/ gives the setup for yours. Clients that read an mcpServers config with a url key, such as Cursor, take the snippet below. VS Code uses a servers key with "type": "http" in .vscode/mcp.json, and Claude Desktop adds remote servers as connectors in its settings.

{
  "mcpServers": {
    "google-news": {
      "url": "https://mcp.apify.com?tools=zovo-one/google-news-scraper"
    }
  }
}

On the first connection your browser opens so you can sign in to Apify (OAuth). If your client can't do that, add a header instead: "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }.

Then ask in plain words, for example "Find news about the EU AI Act from the last 3 days in the UK and German editions, and list the publisher links". The agent fills in queries, editions and dateFrom. Each row has a direct publisher URL, so the agent can read the article with a fetch tool next, without going through a Google redirect.

The Actor runs with limited permissions, which the Apify MCP server needs to call it. It's priced per event, so it also works with the agent payment options of the MCP server.

How it works

1. The Actor reads Google News RSS: news.google.com/rss/search?q=... for searches and the topic feeds for topics, with the edition's hl, gl and ceid values.
2. With a date filter, the search gets Google's after: / before: operators. These count whole days in US Pacific time. So the Actor asks for whole Pacific days that cover your range, then keeps only articles whose publishedAt is inside your exact UTC range. Topic feeds have no date parameter, so for them the filter only applies to the articles the feed already holds.
3. Google returns at most 100 articles per request. When "Max articles" is above 100, each search is read one Pacific day at a time, newest day first, back to "From date" (or up to 365 days back). It stops early after 30 days in a row where Google had nothing new for that search (60 days when "From date" is set). Days whose articles were all saved already by another query or edition don't count toward that stop. Day-by-day search covers at most the newest 400 days of the range per query, and a run reads at most 5,000 feed pages (these requests aren't billed, so they're capped). If one day alone has 100 results, some articles of that day may be missing. The run report lists those days under fullDays, and the log says so. A narrower query gets them.
4. Older Google article IDs contain the publisher URL and are decoded with no request. Current IDs need two steps. The Actor loads the article's Google page to read a signature, then asks Google's own URL lookup (batchexecute) for the publisher URL, 10 articles per lookup. That's about 1.1 requests per article. In the Apify run above, 25 articles took 28 resolution requests.
5. Duplicates are removed before charging.

Blocks and proxies

• By default the Actor connects directly. Requests are spaced out, with at least 0.7 s between feed requests and 0.4 s between resolution requests (0.3 s and 0.2 s with a proxy). If Google keeps blocking your runs, turn on Apify Proxy (datacenter). The RESIDENTIAL proxy group is refused while "Resolve publisher URLs" is on, because each resolution downloads a Google page of about 120 KB (compressed).
• Feeds get up to 4 retries and resolutions up to 2, with exponential backoff. Retries happen on HTTP 429 and 403, captcha pages, HTTP 408, 425, 500, 502, 503 and 504, timeouts and network errors. With Apify Proxy, each block switches to a new proxy session.
• After 5 failed URL resolutions in a row (a success resets the count), the rest of the run skips resolution. Articles are still saved, with resolveStatus: "skipped", and aren't charged as url-resolved.
• After 3 feeds in a row fail because of the connection (blocks, consent or non-RSS pages, 5xx answers, timeouts, network errors), the run stops and the remaining feeds are marked skipped. A 400 or 404 for one feed, such as an unknown topic ID, is reported for that feed and doesn't count, so the next feeds still run.
• On the Apify platform, when no proxy is set and Google refuses the direct connection on a feed before that feed saved anything (a block, a cookie consent redirect or a page that isn't RSS), the run switches once to Apify Proxy (datacenter) and retries that feed. This only works when Apify Proxy is available to the run. OUTPUT shows proxyFallbackUsed: true when it happened.

Run report (OUTPUT)

The key-value store record OUTPUT has totals (articlesSaved, urlsResolved, duplicatesSkipped, filteredOutByDate, stoppedEarly, resolutionStopped, proxyFallbackUsed), request counts, the settings used, and one entry per feed (query or topic, per edition). Each entry has articlesSaved, urlsResolved, resolveFailed, duplicatesSkipped, filteredOut, feedRequests, fullDays, error and skipped.

If the platform restarts a run, the articles already saved are skipped and not charged again.

Limits

• Google decides what each feed contains. Search feeds hold at most 100 articles per request, and topic feeds hold about 30 to 100. publishedAt is the time Google lists for the article. Some publishers give only a date, which appears as a fixed time of day.
• There's no article text, snippet, image or author, because Google News RSS doesn't include them. relatedArticles keep Google links (they aren't resolved or charged).
• URL resolution depends on Google's current page format and lookup call. If Google changes them, rows still save with resolveStatus: "failed" and a resolveError, and aren't charged as url-resolved.
• The Actor sends a fixed cookie consent value. If Google stops accepting it, feeds from some IPs (mostly in the EU) end with "Google asked for cookie consent". The error message then suggests Apify Proxy (datacenter) with a US IP.
• A run takes up to 100 queries, 50 topics and 20 editions, and at most 500 feeds (queries and topics times editions). Duplicate removal works inside one run, so a scheduled run doesn't know what the previous run saved.

Is it legal?

The Actor reads public Google News feeds and pages, with no login. It collects metadata (titles, publisher names, dates, links), not article text. Each Google News feed carries this notice: "This XML feed is made available solely for the purpose of rendering Google News results within a personal feed reader for personal, non-commercial use." Publishers own their headlines. Check that your use (for example, republishing headlines) is allowed where you are. This isn't legal advice.

FAQ

Why are there fewer articles than "Max articles"?

The feed had fewer articles, or some were outside your dates or were duplicates. See filteredOut and duplicatesSkipped per feed in OUTPUT.

Can I get only the Google links?

Yes. Turn off "Resolve publisher URLs". Each article then costs $0.0015 and the run is faster.

How do I monitor a topic every hour without paying twice?

Schedule the Actor in Apify and set "From date" a bit wider than the interval, for example 2 hours for an hourly schedule. Duplicate removal only works inside one run, so drop the few repeats by articleId on your side.

Can I get the full article text?

Not from this Actor. Google News feeds don't contain it. Pass publisherUrl to a page fetcher such as Web Fetch to Markdown to get the text.

Which editions exist?

All 85 are in the Editions list in the input form, as COUNTRY:language (for example CA:fr, MX:es-419, TW:zh-Hant).

Why did a row come back with resolveStatus: "failed"?

Google didn't give a publisher URL for that article, or the lookup was blocked. resolveError says which. The row is still saved and charged as an article, but not as url-resolved.

Support Zovo

Zovo builds and maintains this Actor. If it saves you time, you can support the work on GitHub Sponsors. Sponsorship is optional, and the tool works the same without it.

Zovo is made by theluckystrike on GitHub.

More tools by Zovo

• Website Screenshot API. Saves full-page screenshots and PDFs of public web pages.
• YouTube Transcript Scraper. Gets YouTube transcripts and captions from videos, playlists and channels.
• Google Trends Scraper. Gets Google Trends interest over time, by region and related queries, with no browser.
• Google Ads Transparency Scraper. Gets the ads that an advertiser runs on Google, one row per ad.