# Google News Scraper: Real Article URLs, Full Text (`arcadium/google-news`) Actor

Google News articles (headline, publisher, decoded publisher URL, date, snippet, image, story cluster) by keyword or topic; $3.60 per 1,000 articles. Article text $6.00 per 1,000. Any language, monitor mode, free error rows.

- **URL**: https://apify.com/arcadium/google-news.md
- **Developed by:** [Arcadium Data](https://apify.com/arcadium) (community)
- **Categories:** News, AI
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.90 / 1,000 articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google News Scraper: Real Article URLs, Full Text

Give it search terms, Google News topics, top stories or a city, in any language and country, and it returns one JSON row per article: headline, publisher, the publisher's real article URL, publish time, snippet, thumbnail, and the story cluster with other outlets covering the same story. $3.60 per 1,000 articles; you pay only for rows you get.

**Why this actor**

- Real publisher URLs, decoded from Google's `news.google.com/rss/articles/...` links, on by default and included in the price.
- Optional main article text (`includeArticleText`, $6.00 per 1,000) where the publisher serves it openly. Paywalled and blocked pages are flagged, never worked around.
- Monitor mode skips articles you already have.
- Errors come back as free rows that say why.

Built by Arcadium Data: public data, dated and agent-ready, never personal data.

### Use with AI agents

The actor takes plain JSON and returns one flat row per article, so an AI agent or MCP client (for example through [mcp.apify.com](https://mcp.apify.com)) can call `arcadium/google-news` directly. Input example for "Latest news on a company, last 24 hours":

```json
{ "queries": ["OpenAI"], "timeRange": "1d", "maxArticlesPerQuery": 20 }
```

"Full text of today's technology headlines, one row per story, for a RAG index":

```json
{ "topics": ["TECHNOLOGY"], "timeRange": "1d", "dedupeStories": true, "includeArticleText": true, "maxArticlesPerQuery": 30 }
```

"German news about electric cars from two outlets, only what is new since the last run" (schedule it hourly):

```json
{ "queries": ["Elektroauto"], "language": "de", "country": "DE", "sites": ["spiegel.de", "faz.net"], "onlyNew": true, "stateKey": "ev-de" }
```

Read `title`, `url`, `publisher`, `publishedAt` and `snippet`; `text` holds the article body when you asked for it (`textStatus` says `extracted`, `paywalled`, `blocked` or `too_short`). Articles with the same `storyId` are the same story from different outlets. 100 articles cost $0.36; 100 with full text $0.60. A query that finds nothing or a feed that cannot be read comes back as a free row with `rowType: "error"`, a `code` (`no_match`, `not_found`, `blocked`, `unavailable`, `invalid_input`) and a `message`, never as a failed run, so an agent can tell "no news" from "try again later".

### What you get

| Field | What it holds |
| --- | --- |
| `articleId`, `title` | Google's article id (stable, use it to dedupe) and the headline without the " - Publisher" suffix. |
| `publisher`, `publisherDomain`, `publisherUrl` | The outlet: name, domain (`reuters.com`) and home page. |
| `url`, `urlDecoded` | The publisher's own article URL, decoded from Google's redirect link. |
| `googleNewsUrl` | Google's link to the article. |
| `publishedAt` | Publish time, ISO 8601 UTC. |
| `snippet`, `thumbnailUrl` | The article's own summary and lead image, read from its page. |
| `storyId`, `isStoryLead`, `relatedCount`, `relatedCoverage` | Story cluster: the same story across outlets, with the other outlets' articles listed. |
| `textStatus`, `text`, `wordCount`, `isPaywalled`, `pageLanguage` | Full text and how reading it went (with `includeArticleText`). |
| `source`, `sourceType`, `position`, `edition` | Which query, section, location or URL the article came from, its rank there, and the edition (`US:en`). |
| `scrapedAt` | When the row was read. |

Output example:

```json
{
  "rowType": "article",
  "platform": "google-news",
  "articleId": "CBMiggFBVV95cUxPYXI4MDQ5UDdXZDNuUUpwYkVwY3RYM05lZzhqcFhlb0NfeDVacEFfYkNQWkRuNDZPM05pRndYcjZ2QXBxeHA1ZnpaSVZjNEhXRnZvdkhTZTR3aFppbVlxUXJpeHRhSXRvV0R1VFRuUmhNeTdmdkJ5ZVNmSGFmWlNPbjZ3",
  "title": "Southwest Airlines beats estimates as fuel costs ease",
  "publisher": "CNBC",
  "publisherDomain": "cnbc.com",
  "publisherUrl": "https://www.cnbc.com",
  "url": "https://www.cnbc.com/2026/10/21/southwest-airlines-luv-3q-2026-earnings.html",
  "urlDecoded": true,
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMiggFBVV95cUxPYXI4MDQ5UDdX...?oc=5",
  "publishedAt": "2026-10-21T11:02:00.000Z",
  "snippet": "Southwest Airlines reported third-quarter results that topped expectations ...",
  "thumbnailUrl": "https://image.cnbcfm.com/api/v1/image/108444221-1792627087371.jpg",
  "storyId": "3f9a1c0d52e8b7a4",
  "isStoryLead": true,
  "relatedCount": 4,
  "relatedCoverage": [
    {
      "articleId": "CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE",
      "title": "Southwest earnings top forecasts",
      "publisher": "Reuters",
      "googleNewsUrl": "https://news.google.com/rss/articles/CBMie0FVX3lxTE5aVW9GbzJYU2tyRUZaZXhuYVZLUzNKaURYMF96UUxGelZE?oc=5"
    }
  ],
  "isPaywalled": false,
  "textStatus": "extracted",
  "text": "Southwest Airlines on Wednesday reported third-quarter earnings ...",
  "wordCount": 812,
  "pageLanguage": "en",
  "source": "BUSINESS",
  "sourceType": "section",
  "position": 1,
  "edition": "US:en",
  "scrapedAt": "2026-10-22T08:00:00.000Z"
}
```

### Input

```json
{
  "queries": ["artificial intelligence", "\"interest rates\" -opinion"],
  "topics": ["BUSINESS"],
  "language": "en",
  "country": "US",
  "timeRange": "7d",
  "maxArticlesPerQuery": 50
}
```

| Field | What it does |
| --- | --- |
| `queries` | Search terms. Google News operators work: `"exact phrase"`, `OR`, `-word`, `intitle:`. |
| `topics` | Sections: `WORLD`, `NATION`, `BUSINESS`, `TECHNOLOGY`, `ENTERTAINMENT`, `SPORTS`, `SCIENCE`, `HEALTH`. |
| `topStories` | The edition's Top stories page. |
| `locations` | Local news for a city or region, e.g. `"Berlin"`. |
| `startUrls` | news.google.com search, topic, story or publication URLs (the edition in the URL is kept). |
| `language`, `country` | The edition, e.g. `de` + `DE`, `pt-BR` + `BR`, `ja` + `JP`. An edition id such as `US:en` works too. |
| `sites`, `excludeWords` | Only these publishers (`reuters.com`); leave out words. |
| `timeRange` | `1h`, `1d`, `7d`, `30d`, `1y` or `any`. |
| `dateFrom`, `dateTo` | Exact range, `YYYY-MM-DD` or relative (`"7 days"`). |
| `maxArticlesPerQuery` | Articles per query or section (default 100, up to 5,000; above ~100 a search is split into one-day windows). |
| `maxItems` | Stop after this many articles in total. A cost cap. |
| `decodeUrls`, `includeSnippets` | Publisher URL decoding and snippet/thumbnail (both on by default). |
| `includeArticleText` | Main text of each article (charged as `article-text`). |
| `dedupeStories`, `includeRelated` | One row per story; list related coverage (on). |
| `onlyNew`, `stateKey` | Monitor mode: only articles not returned by an earlier run with the same key. |
| `includeErrorRows` | A free row for every input without data (on by default). |

Input names of the most used Google News scrapers are accepted as they are (`keywords`, `maxArticles`, `timeframe`, `region_language`, `extractImages`, `query`, `hl`, `gl`, `time_period`, `topic`, `publishedAfter`, `startInputs`, `siteFilter`, `regionLanguage`, `resolveUrls`, `enrichBody`, `searchTerms`, `includeTopHeadlines`, `resolveArticleUrls`, `includeArticleContent`, `monitorMode`, `monitorKey`, `fetchArticlePage`, `dateFrom`, `dateTo`), so an existing input can be pasted in.

#### How URL decoding and full text work

Google News links point to `news.google.com/rss/articles/...`, not to the publisher. The actor resolves them the way the Google News page itself does when you click a link, batched (about 20 articles per decode call), and keeps decoded URLs in a store in your account so the same article is never decoded twice. With `includeArticleText` it then reads the publisher's page once and extracts the main text (Readability-style), dropping navigation, ads, related links and bylines. If the publisher marks the article as not free to read (`isAccessibleForFree: false`, a locked content tier) or shows a paywall, the text is left out and `textStatus` is `paywalled`; if the site refuses automated visits, it is `blocked`. Both are charged as a plain `article`, not `article-text`.

#### Monitor mode

`onlyNew: true` remembers, per query and edition, which of the top `maxArticlesPerQuery` articles it has already returned (key `stateKey`). Schedule the actor every hour or day and each run returns only the new ones. A run with nothing new ends without rows and without errors.

### Use it from code

Python:

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("arcadium/google-news").call(run_input={
    "queries": ["nvidia"],
    "timeRange": "1d",
    "maxArticlesPerQuery": 50,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["rowType"] == "article":
        print(item["publishedAt"][:16], item["publisher"], item["title"], item["url"])
```

JavaScript:

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('arcadium/google-news').call({
  topics: ['HEALTH'],
  language: 'en',
  country: 'GB',
  includeArticleText: true,
  maxArticlesPerQuery: 20,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.filter((i) => i.textStatus === 'extracted').length, 'articles with text');
```

### Pricing

You pay per article delivered, at the price of your Apify plan. Error rows, repeats and articles outside your date range are free.

| Apify plan | Per 1,000 articles | Per 1,000 articles with full text |
| --- | --- | --- |
| Free | $3.60 | $6.00 |
| Starter (Bronze) | $3.60 | $5.25 |
| Scale (Silver) | $3.38 | $4.45 |
| Business (Gold) | $0.90 | $2.50 |
| Platinum | $0.90 | $2.50 |
| Diamond | $0.90 | $2.50 |

Plus a one-off $0.001 per GB of memory when a run starts. URL decoding, snippets, images and story clusters are included. Articles whose text is paywalled or blocked are charged at the article price.

Set a maximum cost per run in the Apify run options and the actor stops before it goes over.

### Related actors

- [Google Trends Scraper: Interest, Related Queries, Trending](https://apify.com/arcadium/google-trends)
- [YouTube Transcript & Subtitles Scraper: Text, SRT, VTT](https://apify.com/arcadium/youtube-transcripts)

### Use cases

- News monitoring and alerts: every new article on your brand, competitors or topics, hourly, in any language.
- PR and media tracking: which outlets covered a story (`storyId`, `relatedCoverage`) and when.
- Market and investment research: company and sector news with real URLs and full text for analysis.
- AI agents and RAG: "latest news on X" with clean text and sources, one row per story.
- Dataset building: headlines by topic and country over time.

### Limitations

- About 100 articles per query per Google News page; bigger requests are split into one-day windows, so very broad searches over long ranges are sampled per day.
- Snippets, thumbnails and text come from the publisher's page. Some large sites refuse automated visits (for example nytimes.com and reuters.com); their rows keep the headline and decoded URL, and `textStatus` says `blocked`.
- Paywalled articles get no text, by design.
- Related coverage lists other outlets' headlines and Google links; their publisher URLs are not decoded. Turn `dedupeStories` off to get those articles as rows of their own (with decoded URLs) when the feed lists them.
- Google News ranking and clusters are Google's; publish times are as Google reports them.

### Privacy

Public data only: Google News pages and feeds as an anonymous visitor sees them, and the publishers' public article pages. No login, no account, no cookies beyond Google's anonymous consent cookie. Author bylines are not collected; emails and phone numbers in snippets and text are replaced with `[email]` and `[phone]`. Article text can mention people in the news, as the article itself does.

### FAQ

**Why are some URLs null?**
`decodeUrls` was off, or Google did not resolve that link in this run. `googleNewsUrl` always works in a browser.

**Why is the text empty for some articles?**
Look at `textStatus`: `paywalled` (the publisher marks it as subscriber content), `blocked` (the site refused the visit), `too_short` (no article text on the page, e.g. a video or live page).

**How do I get only new articles every hour?**
Set `onlyNew: true` and a `stateKey`, then schedule the actor. Each article comes once.

**Something looks wrong in the output. Who do I tell?**
Open an issue on the actor page with the run link. We read them all.

# Actor input Schema

## `queries` (type: `array`):

Google News search terms, e.g. \["openai", "electric cars"]. Google News operators work: "exact phrase", OR, -word, intitle:word. Each query gives up to maxArticlesPerQuery articles, newest and most relevant first.

## `topics` (type: `array`):

Google News sections to read, e.g. BUSINESS or TECHNOLOGY. Section stories come grouped: each article lists the other outlets covering the same story (relatedCoverage). For a custom topic page use its news.google.com/topics/... URL in startUrls.

## `topStories` (type: `boolean`):

Also read the Top stories page of the edition (the Google News front page), grouped by story with related coverage.

## `locations` (type: `array`):

Local headlines for a place, e.g. \["Berlin", "Austin, Texas"], as on the Google News local section.

## `startUrls` (type: `array`):

news.google.com search, topic, story or publication URLs, with or without /rss, e.g. https://news.google.com/topics/CAAqJggKIiBDQkFTRWdvSUwyMHZNRGx6TVdZU0FtVnVHZ0pWVXlnQVAB?hl=en-US\&gl=US\&ceid=US:en. The edition in the URL (hl, gl, ceid) is kept.

## `sites` (type: `array`):

Publisher domains to keep, e.g. \["reuters.com", "apnews.com"]. Searches send them to Google as site: filters; sections and top stories are filtered by the publisher's domain.

## `excludeWords` (type: `array`):

Words that must not appear in search results, e.g. \["opinion", "podcast"] (sent as -word). Searches only.

## `language` (type: `string`):

Edition language, e.g. en, de, fr, es, ja, pt-BR, zh-TW (default en). An edition id such as US:en or BR:pt-419 is accepted too and also sets the country.

## `country` (type: `string`):

Edition country as a two-letter code, e.g. US, GB, DE, IN, BR (default US). Google News picks local outlets and ranking for it.

## `timeRange` (type: `string`):

Only articles published within this window before the run. For searches Google applies it; for sections and top stories it is filtered by the published date. A date range below wins over it.

## `dateFrom` (type: `string`):

Date as YYYY-MM-DD (e.g. 2026-10-01) or relative (e.g. 7 days). Searches send it as after:; other feeds are filtered by date.

## `dateTo` (type: `string`):

Date as YYYY-MM-DD (e.g. 2026-10-21) or relative (e.g. 1 day), inclusive. Searches send it as before: the next day.

## `maxArticlesPerQuery` (type: `integer`):

Articles per query, section, location or URL, 1 to 5,000 (default 100). One Google News page holds about 100; above that a search is split into one-day windows across the date range (default the last 30 days).

## `maxItems` (type: `integer`):

Stops the run after this many articles across all inputs, 1 to 1,000,000. A cost cap; leave empty for no limit.

## `decodeUrls` (type: `boolean`):

Turn Google's news.google.com/rss/articles/... redirect links into the publisher's real article URL (on by default). Off: faster, url stays null and only googleNewsUrl is set.

## `includeSnippets` (type: `boolean`):

Read each article's page for its summary (snippet) and lead image (thumbnailUrl), on by default. Needs decodeUrls. Pages that block automated visits leave both empty.

## `includeArticleText` (type: `boolean`):

Also extract the main text of each article (charged as article-text). Only text the publisher serves openly: paywalled or blocked pages are flagged in textStatus and charged as a plain article, never worked around. Author names are not included.

## `includeRelated` (type: `boolean`):

List the other outlets' articles on the same story in relatedCoverage (on by default). Off keeps only relatedCount.

## `dedupeStories` (type: `boolean`):

Keep only the lead article of each story; the other outlets' versions are listed in its relatedCoverage instead of getting their own rows. Fewer rows, lower cost.

## `onlyNew` (type: `boolean`):

Return only articles not returned by an earlier run with the same stateKey, per query and edition. Schedule the actor and get each new article once. The first run returns the current list.

## `stateKey` (type: `string`):

Name of the saved monitor state, e.g. brand-watch (1-40 letters, digits or dashes; default `default`). Use different keys for independent monitors with the same queries.

## `includeErrorRows` (type: `boolean`):

Push a free row with rowType error (code and message) for every query or URL that gave nothing, so you can tell 'no results' from 'blocked'.

## `maxRetries` (type: `integer`):

Retries for a Google News request that fails or is rate limited, 0 to 10 (default 4). Each retry uses a new IP.

## `proxyConfiguration` (type: `object`):

Advanced: leave empty. The actor uses Apify's datacenter proxy by itself. Only your own proxy URLs (proxyUrls) are used if set; Apify proxy groups are ignored.

## Actor input object example

```json
{
  "queries": [
    "artificial intelligence"
  ],
  "language": "en",
  "country": "US",
  "maxArticlesPerQuery": 20,
  "maxItems": 50,
  "includeErrorRows": true
}
```

# Actor output Schema

## `results` (type: `string`):

All rows of this run in the default dataset.

## `view_overview` (type: `string`):

No description

## `view_text` (type: `string`):

No description

## `view_stories` (type: `string`):

No description

## `view_errors` (type: `string`):

No description

## `record_stats` (type: `string`):

Requests, data volume, retries, URL decoding counts, proxy tiers and cost per article for this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ],
    "language": "en",
    "country": "US",
    "maxArticlesPerQuery": 20,
    "maxItems": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("arcadium/google-news").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["artificial intelligence"],
    "language": "en",
    "country": "US",
    "maxArticlesPerQuery": 20,
    "maxItems": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("arcadium/google-news").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ],
  "language": "en",
  "country": "US",
  "maxArticlesPerQuery": 20,
  "maxItems": 50
}' |
apify call arcadium/google-news --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arcadium/google-news"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/eTiHOfBU4jn2fUKCS/builds/qufIIXhD2dT5Xi3OQ/openapi.json
