Google News Scraper: Real Article URLs, Full Text
Pricing
from $0.90 / 1,000 articles
Google News Scraper: Real Article URLs, Full Text
Google News articles (headline, publisher, decoded publisher URL, date, snippet, image, story cluster) by keyword or topic; $3.60 per 1,000 articles. Article text $6.00 per 1,000. Any language, monitor mode, free error rows.
Pricing
from $0.90 / 1,000 articles
Rating
0.0
(0)
Developer
Arcadium Data
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
17 days ago
Last modified
Features
Share
Google News Scraper: Real Article URLs, Full Text
Give it search terms, Google News topics, top stories or a city, in any language and country, and it returns one JSON row per article: headline, publisher, the publisher's real article URL, publish time, snippet, thumbnail, and the story cluster with other outlets covering the same story. $3.60 per 1,000 articles; you pay only for rows you get.
Why this actor
- Real publisher URLs, decoded from Google's
news.google.com/rss/articles/...links, on by default and included in the price. - Optional main article text (
includeArticleText, $6.00 per 1,000) where the publisher serves it openly. Paywalled and blocked pages are flagged, never worked around. - Monitor mode skips articles you already have.
- Errors come back as free rows that say why.
Built by Arcadium Data: public data, dated and agent-ready, never personal data.
Use with AI agents
The actor takes plain JSON and returns one flat row per article, so an AI agent or MCP client (for example through mcp.apify.com) can call arcadium/google-news directly. Input example for "Latest news on a company, last 24 hours":
{ "queries": ["OpenAI"], "timeRange": "1d", "maxArticlesPerQuery": 20 }
"Full text of today's technology headlines, one row per story, for a RAG index":
{ "topics": ["TECHNOLOGY"], "timeRange": "1d", "dedupeStories": true, "includeArticleText": true, "maxArticlesPerQuery": 30 }
"German news about electric cars from two outlets, only what is new since the last run" (schedule it hourly):
{ "queries": ["Elektroauto"], "language": "de", "country": "DE", "sites": ["spiegel.de", "faz.net"], "onlyNew": true, "stateKey": "ev-de" }
Read title, url, publisher, publishedAt and snippet; text holds the article body when you asked for it (textStatus says extracted, paywalled, blocked or too_short). Articles with the same storyId are the same story from different outlets. 100 articles cost $0.36; 100 with full text $0.60. A query that finds nothing or a feed that cannot be read comes back as a free row with rowType: "error", a code (no_match, not_found, blocked, unavailable, invalid_input) and a message, never as a failed run, so an agent can tell "no news" from "try again later".
What you get
| Field | What it holds |
|---|---|
articleId, title | Google's article id (stable, use it to dedupe) and the headline without the " - Publisher" suffix. |
publisher, publisherDomain, publisherUrl | The outlet: name, domain (reuters.com) and home page. |
url, urlDecoded | The publisher's own article URL, decoded from Google's redirect link. |
googleNewsUrl | Google's link to the article. |
publishedAt | Publish time, ISO 8601 UTC. |
snippet, thumbnailUrl | The article's own summary and lead image, read from its page. |
storyId, isStoryLead, relatedCount, relatedCoverage | Story cluster: the same story across outlets, with the other outlets' articles listed. |
textStatus, text, wordCount, isPaywalled, pageLanguage | Full text and how reading it went (with includeArticleText). |
source, sourceType, position, edition | Which query, section, location or URL the article came from, its rank there, and the edition (US:en). |
scrapedAt | When the row was read. |
Output example:
{"rowType": "article","platform": "google-news","articleId": "CBMiggFBVV95cUxPYXI4MDQ5UDdXZDNuUUpwYkVwY3RYM05lZzhqcFhlb0NfeDVacEFfYkNQWkRuNDZPM05pRndYcjZ2QXBxeHA1ZnpaSVZjNEhXRnZvdkhTZTR3aFppbVlxUXJpeHRhSXRvV0R1VFRuUmhNeTdmdkJ5ZVNmSGFmWlNPbjZ3","title": "Southwest Airlines beats estimates as fuel costs ease","publisher": "CNBC","publisherDomain": "cnbc.com","publisherUrl": "https://www.cnbc.com","url": "https://www.cnbc.com/2026/10/21/southwest-airlines-luv-3q-2026-earnings.html","urlDecoded": true,"googleNewsUrl": "https://news.google.com/rss/articles/CBMiggFBVV95cUxPYXI4MDQ5UDdX...?oc=5","publishedAt": "2026-10-21T11:02:00.000Z","snippet": "Southwest Airlines reported third-quarter results that topped expectations ...","thumbnailUrl": "https://image.cnbcfm.com/api/v1/image/108444221-1792627087371.jpg","storyId": "3f9a1c0d52e8b7a4","isStoryLead": true,"relatedCount": 4,"relatedCoverage": [{"articleId": "CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE","title": "Southwest earnings top forecasts","publisher": "Reuters","googleNewsUrl": "https://news.google.com/rss/articles/CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE?oc=5"}],"isPaywalled": false,"textStatus": "extracted","text": "Southwest Airlines on Wednesday reported third-quarter earnings ...","wordCount": 812,"pageLanguage": "en","source": "BUSINESS","sourceType": "section","position": 1,"edition": "US:en","scrapedAt": "2026-10-22T08:00:00.000Z"}
Input
{"queries": ["artificial intelligence", "\"interest rates\" -opinion"],"topics": ["BUSINESS"],"language": "en","country": "US","timeRange": "7d","maxArticlesPerQuery": 50}
| Field | What it does |
|---|---|
queries | Search terms. Google News operators work: "exact phrase", OR, -word, intitle:. |
topics | Sections: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH. |
topStories | The edition's Top stories page. |
locations | Local news for a city or region, e.g. "Berlin". |
startUrls | news.google.com search, topic, story or publication URLs (the edition in the URL is kept). |
language, country | The edition, e.g. de + DE, pt-BR + BR, ja + JP. An edition id such as US:en works too. |
sites, excludeWords | Only these publishers (reuters.com); leave out words. |
timeRange | 1h, 1d, 7d, 30d, 1y or any. |
dateFrom, dateTo | Exact range, YYYY-MM-DD or relative ("7 days"). |
maxArticlesPerQuery | Articles per query or section (default 100, up to 5,000; above ~100 a search is split into one-day windows). |
maxItems | Stop after this many articles in total. A cost cap. |
decodeUrls, includeSnippets | Publisher URL decoding and snippet/thumbnail (both on by default). |
includeArticleText | Main text of each article (charged as article-text). |
dedupeStories, includeRelated | One row per story; list related coverage (on). |
onlyNew, stateKey | Monitor mode: only articles not returned by an earlier run with the same key. |
includeErrorRows | A free row for every input without data (on by default). |
Input names of the most used Google News scrapers are accepted as they are (keywords, maxArticles, timeframe, region_language, extractImages, query, hl, gl, time_period, topic, publishedAfter, startInputs, siteFilter, regionLanguage, resolveUrls, enrichBody, searchTerms, includeTopHeadlines, resolveArticleUrls, includeArticleContent, monitorMode, monitorKey, fetchArticlePage, dateFrom, dateTo), so an existing input can be pasted in.
How URL decoding and full text work
Google News links point to news.google.com/rss/articles/..., not to the publisher. The actor resolves them the way the Google News page itself does when you click a link, batched (about 20 articles per decode call), and keeps decoded URLs in a store in your account so the same article is never decoded twice. With includeArticleText it then reads the publisher's page once and extracts the main text (Readability-style), dropping navigation, ads, related links and bylines. If the publisher marks the article as not free to read (isAccessibleForFree: false, a locked content tier) or shows a paywall, the text is left out and textStatus is paywalled; if the site refuses automated visits, it is blocked. Both are charged as a plain article, not article-text.
Monitor mode
onlyNew: true remembers, per query and edition, which of the top maxArticlesPerQuery articles it has already returned (key stateKey). Schedule the actor every hour or day and each run returns only the new ones. A run with nothing new ends without rows and without errors.
Use it from code
Python:
from apify_client import ApifyClientclient = ApifyClient("<YOUR_APIFY_TOKEN>")run = client.actor("arcadium/google-news").call(run_input={"queries": ["nvidia"],"timeRange": "1d","maxArticlesPerQuery": 50,})for item in client.dataset(run["defaultDatasetId"]).iterate_items():if item["rowType"] == "article":print(item["publishedAt"][:16], item["publisher"], item["title"], item["url"])
JavaScript:
import { ApifyClient } from 'apify-client';const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });const run = await client.actor('arcadium/google-news').call({topics: ['HEALTH'],language: 'en',country: 'GB',includeArticleText: true,maxArticlesPerQuery: 20,});const { items } = await client.dataset(run.defaultDatasetId).listItems();console.log(items.filter((i) => i.textStatus === 'extracted').length, 'articles with text');
Pricing
You pay per article delivered, at the price of your Apify plan. Error rows, repeats and articles outside your date range are free.
| Apify plan | Per 1,000 articles | Per 1,000 articles with full text |
|---|---|---|
| Free | $3.60 | $6.00 |
| Starter (Bronze) | $3.60 | $5.25 |
| Scale (Silver) | $3.38 | $4.45 |
| Business (Gold) | $0.90 | $2.50 |
| Platinum | $0.90 | $2.50 |
| Diamond | $0.90 | $2.50 |
Plus a one-off $0.001 per GB of memory when a run starts. URL decoding, snippets, images and story clusters are included. Articles whose text is paywalled or blocked are charged at the article price.
Set a maximum cost per run in the Apify run options and the actor stops before it goes over.
Related actors
- Google Trends Scraper: Interest, Related Queries, Trending
- YouTube Transcript & Subtitles Scraper: Text, SRT, VTT
Use cases
- News monitoring and alerts: every new article on your brand, competitors or topics, hourly, in any language.
- PR and media tracking: which outlets covered a story (
storyId,relatedCoverage) and when. - Market and investment research: company and sector news with real URLs and full text for analysis.
- AI agents and RAG: "latest news on X" with clean text and sources, one row per story.
- Dataset building: headlines by topic and country over time.
Limitations
- About 100 articles per query per Google News page; bigger requests are split into one-day windows, so very broad searches over long ranges are sampled per day.
- Snippets, thumbnails and text come from the publisher's page. Some large sites refuse automated visits (for example nytimes.com and reuters.com); their rows keep the headline and decoded URL, and
textStatussaysblocked. - Paywalled articles get no text, by design.
- Related coverage lists other outlets' headlines and Google links; their publisher URLs are not decoded. Turn
dedupeStoriesoff to get those articles as rows of their own (with decoded URLs) when the feed lists them. - Google News ranking and clusters are Google's; publish times are as Google reports them.
Privacy
Public data only: Google News pages and feeds as an anonymous visitor sees them, and the publishers' public article pages. No login, no account, no cookies beyond Google's anonymous consent cookie. Author bylines are not collected; emails and phone numbers in snippets and text are replaced with [email] and [phone]. Article text can mention people in the news, as the article itself does.
FAQ
Why are some URLs null?
decodeUrls was off, or Google did not resolve that link in this run. googleNewsUrl always works in a browser.
Why is the text empty for some articles?
Look at textStatus: paywalled (the publisher marks it as subscriber content), blocked (the site refused the visit), too_short (no article text on the page, e.g. a video or live page).
How do I get only new articles every hour?
Set onlyNew: true and a stateKey, then schedule the actor. Each article comes once.
Something looks wrong in the output. Who do I tell? Open an issue on the actor page with the run link. We read them all.