Google News Scraper: Real Article URLs, Full Text avatar

Google News Scraper: Real Article URLs, Full Text

Pricing

from $0.90 / 1,000 articles

Go to Apify Store
Google News Scraper: Real Article URLs, Full Text

Google News Scraper: Real Article URLs, Full Text

Google News articles (headline, publisher, decoded publisher URL, date, snippet, image, story cluster) by keyword or topic; $3.60 per 1,000 articles. Article text $6.00 per 1,000. Any language, monitor mode, free error rows.

Pricing

from $0.90 / 1,000 articles

Rating

0.0

(0)

Developer

Arcadium Data

Arcadium Data

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

17 days ago

Last modified

Categories

Features

Limited permissions

Share

Google News Scraper: Real Article URLs, Full Text

Give it search terms, Google News topics, top stories or a city, in any language and country, and it returns one JSON row per article: headline, publisher, the publisher's real article URL, publish time, snippet, thumbnail, and the story cluster with other outlets covering the same story. $3.60 per 1,000 articles; you pay only for rows you get.

Why this actor

  • Real publisher URLs, decoded from Google's news.google.com/rss/articles/... links, on by default and included in the price.
  • Optional main article text (includeArticleText, $6.00 per 1,000) where the publisher serves it openly. Paywalled and blocked pages are flagged, never worked around.
  • Monitor mode skips articles you already have.
  • Errors come back as free rows that say why.

Built by Arcadium Data: public data, dated and agent-ready, never personal data.

Use with AI agents

The actor takes plain JSON and returns one flat row per article, so an AI agent or MCP client (for example through mcp.apify.com) can call arcadium/google-news directly. Input example for "Latest news on a company, last 24 hours":

{ "queries": ["OpenAI"], "timeRange": "1d", "maxArticlesPerQuery": 20 }

"Full text of today's technology headlines, one row per story, for a RAG index":

{ "topics": ["TECHNOLOGY"], "timeRange": "1d", "dedupeStories": true, "includeArticleText": true, "maxArticlesPerQuery": 30 }

"German news about electric cars from two outlets, only what is new since the last run" (schedule it hourly):

{ "queries": ["Elektroauto"], "language": "de", "country": "DE", "sites": ["spiegel.de", "faz.net"], "onlyNew": true, "stateKey": "ev-de" }

Read title, url, publisher, publishedAt and snippet; text holds the article body when you asked for it (textStatus says extracted, paywalled, blocked or too_short). Articles with the same storyId are the same story from different outlets. 100 articles cost $0.36; 100 with full text $0.60. A query that finds nothing or a feed that cannot be read comes back as a free row with rowType: "error", a code (no_match, not_found, blocked, unavailable, invalid_input) and a message, never as a failed run, so an agent can tell "no news" from "try again later".

What you get

FieldWhat it holds
articleId, titleGoogle's article id (stable, use it to dedupe) and the headline without the " - Publisher" suffix.
publisher, publisherDomain, publisherUrlThe outlet: name, domain (reuters.com) and home page.
url, urlDecodedThe publisher's own article URL, decoded from Google's redirect link.
googleNewsUrlGoogle's link to the article.
publishedAtPublish time, ISO 8601 UTC.
snippet, thumbnailUrlThe article's own summary and lead image, read from its page.
storyId, isStoryLead, relatedCount, relatedCoverageStory cluster: the same story across outlets, with the other outlets' articles listed.
textStatus, text, wordCount, isPaywalled, pageLanguageFull text and how reading it went (with includeArticleText).
source, sourceType, position, editionWhich query, section, location or URL the article came from, its rank there, and the edition (US:en).
scrapedAtWhen the row was read.

Output example:

{
"rowType": "article",
"platform": "google-news",
"articleId": "CBMiggFBVV95cUxPYXI4MDQ5UDdXZDNuUUpwYkVwY3RYM05lZzhqcFhlb0NfeDVacEFfYkNQWkRuNDZPM05pRndYcjZ2QXBxeHA1ZnpaSVZjNEhXRnZvdkhTZTR3aFppbVlxUXJpeHRhSXRvV0R1VFRuUmhNeTdmdkJ5ZVNmSGFmWlNPbjZ3",
"title": "Southwest Airlines beats estimates as fuel costs ease",
"publisher": "CNBC",
"publisherDomain": "cnbc.com",
"publisherUrl": "https://www.cnbc.com",
"url": "https://www.cnbc.com/2026/10/21/southwest-airlines-luv-3q-2026-earnings.html",
"urlDecoded": true,
"googleNewsUrl": "https://news.google.com/rss/articles/CBMiggFBVV95cUxPYXI4MDQ5UDdX...?oc=5",
"publishedAt": "2026-10-21T11:02:00.000Z",
"snippet": "Southwest Airlines reported third-quarter results that topped expectations ...",
"thumbnailUrl": "https://image.cnbcfm.com/api/v1/image/108444221-1792627087371.jpg",
"storyId": "3f9a1c0d52e8b7a4",
"isStoryLead": true,
"relatedCount": 4,
"relatedCoverage": [
{
"articleId": "CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE",
"title": "Southwest earnings top forecasts",
"publisher": "Reuters",
"googleNewsUrl": "https://news.google.com/rss/articles/CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE?oc=5"
}
],
"isPaywalled": false,
"textStatus": "extracted",
"text": "Southwest Airlines on Wednesday reported third-quarter earnings ...",
"wordCount": 812,
"pageLanguage": "en",
"source": "BUSINESS",
"sourceType": "section",
"position": 1,
"edition": "US:en",
"scrapedAt": "2026-10-22T08:00:00.000Z"
}

Input

{
"queries": ["artificial intelligence", "\"interest rates\" -opinion"],
"topics": ["BUSINESS"],
"language": "en",
"country": "US",
"timeRange": "7d",
"maxArticlesPerQuery": 50
}
FieldWhat it does
queriesSearch terms. Google News operators work: "exact phrase", OR, -word, intitle:.
topicsSections: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH.
topStoriesThe edition's Top stories page.
locationsLocal news for a city or region, e.g. "Berlin".
startUrlsnews.google.com search, topic, story or publication URLs (the edition in the URL is kept).
language, countryThe edition, e.g. de + DE, pt-BR + BR, ja + JP. An edition id such as US:en works too.
sites, excludeWordsOnly these publishers (reuters.com); leave out words.
timeRange1h, 1d, 7d, 30d, 1y or any.
dateFrom, dateToExact range, YYYY-MM-DD or relative ("7 days").
maxArticlesPerQueryArticles per query or section (default 100, up to 5,000; above ~100 a search is split into one-day windows).
maxItemsStop after this many articles in total. A cost cap.
decodeUrls, includeSnippetsPublisher URL decoding and snippet/thumbnail (both on by default).
includeArticleTextMain text of each article (charged as article-text).
dedupeStories, includeRelatedOne row per story; list related coverage (on).
onlyNew, stateKeyMonitor mode: only articles not returned by an earlier run with the same key.
includeErrorRowsA free row for every input without data (on by default).

Input names of the most used Google News scrapers are accepted as they are (keywords, maxArticles, timeframe, region_language, extractImages, query, hl, gl, time_period, topic, publishedAfter, startInputs, siteFilter, regionLanguage, resolveUrls, enrichBody, searchTerms, includeTopHeadlines, resolveArticleUrls, includeArticleContent, monitorMode, monitorKey, fetchArticlePage, dateFrom, dateTo), so an existing input can be pasted in.

How URL decoding and full text work

Google News links point to news.google.com/rss/articles/..., not to the publisher. The actor resolves them the way the Google News page itself does when you click a link, batched (about 20 articles per decode call), and keeps decoded URLs in a store in your account so the same article is never decoded twice. With includeArticleText it then reads the publisher's page once and extracts the main text (Readability-style), dropping navigation, ads, related links and bylines. If the publisher marks the article as not free to read (isAccessibleForFree: false, a locked content tier) or shows a paywall, the text is left out and textStatus is paywalled; if the site refuses automated visits, it is blocked. Both are charged as a plain article, not article-text.

Monitor mode

onlyNew: true remembers, per query and edition, which of the top maxArticlesPerQuery articles it has already returned (key stateKey). Schedule the actor every hour or day and each run returns only the new ones. A run with nothing new ends without rows and without errors.

Use it from code

Python:

from apify_client import ApifyClient
client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("arcadium/google-news").call(run_input={
"queries": ["nvidia"],
"timeRange": "1d",
"maxArticlesPerQuery": 50,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
if item["rowType"] == "article":
print(item["publishedAt"][:16], item["publisher"], item["title"], item["url"])

JavaScript:

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('arcadium/google-news').call({
topics: ['HEALTH'],
language: 'en',
country: 'GB',
includeArticleText: true,
maxArticlesPerQuery: 20,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.filter((i) => i.textStatus === 'extracted').length, 'articles with text');

Pricing

You pay per article delivered, at the price of your Apify plan. Error rows, repeats and articles outside your date range are free.

Apify planPer 1,000 articlesPer 1,000 articles with full text
Free$3.60$6.00
Starter (Bronze)$3.60$5.25
Scale (Silver)$3.38$4.45
Business (Gold)$0.90$2.50
Platinum$0.90$2.50
Diamond$0.90$2.50

Plus a one-off $0.001 per GB of memory when a run starts. URL decoding, snippets, images and story clusters are included. Articles whose text is paywalled or blocked are charged at the article price.

Set a maximum cost per run in the Apify run options and the actor stops before it goes over.

Use cases

  • News monitoring and alerts: every new article on your brand, competitors or topics, hourly, in any language.
  • PR and media tracking: which outlets covered a story (storyId, relatedCoverage) and when.
  • Market and investment research: company and sector news with real URLs and full text for analysis.
  • AI agents and RAG: "latest news on X" with clean text and sources, one row per story.
  • Dataset building: headlines by topic and country over time.

Limitations

  • About 100 articles per query per Google News page; bigger requests are split into one-day windows, so very broad searches over long ranges are sampled per day.
  • Snippets, thumbnails and text come from the publisher's page. Some large sites refuse automated visits (for example nytimes.com and reuters.com); their rows keep the headline and decoded URL, and textStatus says blocked.
  • Paywalled articles get no text, by design.
  • Related coverage lists other outlets' headlines and Google links; their publisher URLs are not decoded. Turn dedupeStories off to get those articles as rows of their own (with decoded URLs) when the feed lists them.
  • Google News ranking and clusters are Google's; publish times are as Google reports them.

Privacy

Public data only: Google News pages and feeds as an anonymous visitor sees them, and the publishers' public article pages. No login, no account, no cookies beyond Google's anonymous consent cookie. Author bylines are not collected; emails and phone numbers in snippets and text are replaced with [email] and [phone]. Article text can mention people in the news, as the article itself does.

FAQ

Why are some URLs null? decodeUrls was off, or Google did not resolve that link in this run. googleNewsUrl always works in a browser.

Why is the text empty for some articles? Look at textStatus: paywalled (the publisher marks it as subscriber content), blocked (the site refused the visit), too_short (no article text on the page, e.g. a video or live page).

How do I get only new articles every hour? Set onlyNew: true and a stateKey, then schedule the actor. Each article comes once.

Something looks wrong in the output. Who do I tell? Open an issue on the actor page with the run link. We read them all.