# Google News Scraper – Full Text & Direct URLs (`brii3343/google-news-scraper-full-text`) Actor

Scrape Google News by keyword, topic or link: every article with its direct publisher URL (not the Google redirect), source and date. Optional full text, image and author. Any country and language, day-by-day date ranges past the 100-article limit, and an only-new mode for monitoring.

- **URL**: https://apify.com/brii3343/google-news-scraper-full-text.md
- **Developed by:** [Brian Gastaldelli](https://apify.com/brii3343) (community)
- **Categories:** News, Automation, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### Google News Scraper — direct article URLs, full text and only new articles

Search Google News by keyword, topic or link and get every article with its **real publisher URL** (not the `news.google.com` redirect), title, source, publication time and related coverage. Turn on **full text** to also get the article text, description, image and author from the publisher's page. Any country and language, date ranges day by day, all Google News search operators, and an **only new articles** mode for scheduled monitoring.

#### Why this Actor

- **Direct links that work.** Google News hides the publisher URL behind an encoded redirect. This Actor decodes it: in our tests **459 links out of 459** were decoded, and every checked link we could open (28 out of 28) led to the same article as the Google News title.
- **Full text when you need it.** Clean article text in paragraphs, without menus, ads or cookie banners, plus description, image and author. In our tests **86 articles out of 100** across 5 topics came back with full text; the others are sites that block automated access or show only a paywall preview, and they are **not charged**.
- **Only new articles.** Run it every hour or every day with `onlyNew`: articles already delivered by previous runs are skipped and **not charged**. Ideal for alerts, newsletters and monitoring.
- **More than 100 articles.** A Google News search gives at most ~100 articles. With a date range the Actor searches **one day at a time**, so a month of news for a keyword can return thousands of articles, without duplicates.
- **Everything Google News offers:** keywords with operators (`"exact phrase"`, `-exclude`, `OR`, `site:reuters.com`, `intitle:`), topics (Business, Technology, Sports...), top stories, and any Google News link you copy from your browser (topics, sources, full coverage).
- **Fast and cheap.** 20 articles with direct links in a few seconds; 459 articles from 6 searches in 38 seconds. No proxy needed.

#### Use cases

- **Media monitoring and PR**: every article that mentions your brand, competitors or executives, with the real URL and full text.
- **Market and investment research**: news on companies, sectors and commodities (`site:reuters.com oil`), day by day.
- **AI agents, RAG and LLM pipelines**: fresh news with clean full text, ready for summaries and embeddings.
- **Newsletters and news apps**: scheduled runs with `onlyNew` give you only what is new since the last run.
- **Academic and data journalism**: coverage of a topic over weeks or months, per country and language.

#### Input

| Field | Description |
|---|---|
| Search queries | One per line. Google News operators work: `"climate change" -trump`, `site:reuters.com oil`, `tesla when:1d`. |
| Topics | `TOP` (top stories), `WORLD`, `NATION`, `BUSINESS`, `TECHNOLOGY`, `ENTERTAINMENT`, `SPORTS`, `SCIENCE`, `HEALTH`. |
| Google News links | Links copied from news.google.com: searches, topics, sources, full coverage. They keep their own edition. |
| Country, language | Google News edition, e.g. `US` + `en`, `GB` + `en`, `IT` + `it`, `DE` + `de`, `IN` + `hi`, `BR` + `pt`. |
| Time range | Any time, past hour, 24 hours, 7 days, 30 days, year. |
| From date, to date | `YYYY-MM-DD` or a number of days ago (`7`). One search per day in the range (max 2,000 searches per run). |
| Max articles per search | Default 100 (about the maximum Google News gives for one search). `0` = all. |
| Direct publisher URLs | On by default. Off: faster, only the Google News links. |
| Full text | Off by default. On: text, description, image and author from the publisher page. |
| Only new articles | Skip articles delivered by previous runs of the same search (remembered for the last 5,000 per search). |

Example input:

```json
{
  "queries": ["artificial intelligence", "site:reuters.com oil"],
  "topics": ["BUSINESS"],
  "country": "US",
  "language": "en",
  "dateFrom": "7",
  "maxArticles": 50,
  "fetchDetails": true,
  "onlyNew": false
}
```

#### Output

One row per article, in Google News order. Real output (text shortened here):

```json
{
  "type": "article",
  "search": "climate change",
  "searchType": "query",
  "searchDate": null,
  "position": 15,
  "title": "Samuel Alito steps aside in a major climate case amid scrutiny over oil stock holdings",
  "source": "The Guardian",
  "sourceUrl": "https://www.theguardian.com",
  "publishedAt": "2026-09-28T22:42:00.000Z",
  "url": "https://www.theguardian.com/us-news/2026/sep/28/samuel-alito-supreme-court-oil-climate-case",
  "urlStatus": "ok",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMilgFBVV95cUxQZWZ5eDBRbWhNM05xYW5pYkFPd2t...?oc=5",
  "relatedCoverage": [],
  "edition": "US:en",
  "articleId": "CBMilgFBVV95cUxQZWZ5eDBRbWhNM05xYW5pYkFP...",
  "detailsStatus": "ok",
  "description": "Decision from conservative justice comes before court hears case on whether fossil-fuel firms deceived US public",
  "image": "https://i.guim.co.uk/img/media/393aa1aec0328b16c161b03f8f7b247c8ea33f5e/653_0_6827_5464/master/6827.jpg?width=1200&height=630&quality=85&auto=format&fit=crop&precrop=40:21,offset-x50,offset-y0&overlay-align=bottom%2Cleft&overlay-width=100p&overlay-base64=L2ltZy9zdGF0aWMvb3ZlcmxheXMvdGctZGVmYXVsdC5wbmc&enable=upscale&s=176d6afa533623201f0480d79a3582fc",
  "author": "Guardian staff reporter",
  "wordCount": 295,
  "text": "Supreme court justice Samuel Alito recused himself on Monday from a major climate-change case after facing calls to step aside due to stock holdings in oil companies.\n\nA letter posted in the case did not specify a reason, but the conservative justice has previously recused himself from other cases i [...]",
  "scrapedAt": "2026-09-29T05:51:52.759Z"
}

When several outlets cover the same story, `relatedCoverage` lists them (title, source and Google News link).
```

The fields from `detailsStatus` to `text` are there only with *Full text* on. `detailsStatus` tells what happened on the publisher page:

| detailsStatus | Meaning |
|---|---|
| `ok` | Full text extracted (charged as *Article full text*) |
| `paywalled` | The site shows only a subscriber preview (free) |
| `short_text`, `no_text` | The page has almost no article text, e.g. a video or live page (free) |
| `blocked_or_failed`, `blocked`, `not_found`, `site_unreachable`, `not_html` | The site refused or did not answer (free) |
| `not_charged_limit` | Your maximum cost per run was reached: the article is there, the text is not |

Searches that give nothing come back as a `status` row, never charged:

| status | Meaning |
|---|---|
| `no_results` | Google News has no articles for this search, or no new ones since the last run |
| `invalid_search` | Google News does not know this topic, source or link |
| `unsupported_edition` | Google News has no edition for this country and language (it would answer with US news) |
| `invalid_input`, `invalid_url`, `too_many_searches` | Something in the input is not valid, or the date range needs more than 2,000 searches |
| `error` | Google News did not answer after every retry (rare); try again later |

#### Honest limits

- **About 100 articles per search**: that is what Google News gives. Use a date range to get more (one search per day).
- **Days follow US Pacific time**, as Google News does with `after:` and `before:`: a "day" runs from 07:00 to 07:00 UTC.
- **Some sites block automated access** (for example WSJ, NYT, FT, Politico, Bloomberg): about 1 article in 10 with full text on. You still get the article with its direct URL, and the missing text is not charged.
- **Paywalled articles** give only the preview the site shows to everyone; they are marked `paywalled` and not charged.
- Photo captions that sites put inside the article body can appear in `text`.

#### Pricing

Pay per event, only for what you get:

- **Article** (`article`): an article with its direct publisher URL.
- **Article full text** (`article-details`): charged only when the text is actually extracted (`detailsStatus: ok`).
- Articles without a decoded URL, articles skipped by *Only new articles* and status rows are free.

Prices per 1,000 events for each Apify plan are shown in the Pricing tab. The Actor honors your *Maximum cost per run*: near the limit it keeps complete articles (URL and text) and stops.

#### Tips

- For monitoring, schedule the Actor with *Only new articles* on and *Time range* = past 24 hours: each run returns only what is new.
- For history, set *From date* and *To date*: one search per day, up to 100 articles per day per query.
- `site:` limits a search to one publisher, `when:1d` to the last day, `intitle:` to words in the title.
- For another country, change *Country* and *Language* together (`FR` + `fr`, `JP` + `ja`, `MX` + `es`).

#### Other Actors by the same author

- [Google Trends API](https://apify.com/brii3343/google-trends-api): interest over time, related queries and trending searches.
- [YouTube Transcript Scraper](https://apify.com/brii3343/youtube-transcript-scraper): transcripts and subtitles of any YouTube video.

# Actor input Schema

## `queries` (type: `array`):

One search per line, with Google operators if you like: <code>"electric cars"</code> (exact phrase), <code>tesla OR rivian</code>, <code>apple -iphone</code>, <code>site:reuters.com oil</code>, <code>intitle:bitcoin</code>.

## `topics` (type: `array`):

Google News sections: <code>TOP</code> (top stories), <code>WORLD</code>, <code>NATION</code>, <code>BUSINESS</code>, <code>TECHNOLOGY</code>, <code>ENTERTAINMENT</code>, <code>SPORTS</code>, <code>SCIENCE</code>, <code>HEALTH</code>.

## `googleNewsUrls` (type: `array`):

Links copied from news.google.com: a search, a topic or section page, a publication, full coverage, or the home page. Country and language are taken from the link.

## `country` (type: `string`):

Google News edition: two-letter country code (<code>US</code>, <code>GB</code>, <code>IN</code>, <code>DE</code>, <code>IT</code>, <code>BR</code>...).

## `language` (type: `string`):

Language of the edition: <code>en</code>, <code>de</code>, <code>fr</code>, <code>es</code>, <code>it</code>, <code>pt</code>, <code>ja</code>, <code>hi</code>... The country must have an edition in that language (for example <code>CA</code> + <code>fr</code>).

## `timeRange` (type: `string`):

Only articles published in the last hour, day, week, month or year (search queries only). Ignored when you set a date range.

## `dateFrom` (type: `string`):

Search every day from this date to the "To date" (one search per day, so you get up to ~100 articles per day instead of 100 in total). YYYY-MM-DD, or a number of days ago (<code>7</code>).

## `dateTo` (type: `string`):

Last day of the date range (default: today). YYYY-MM-DD, or a number of days ago (<code>0</code> = today).

## `maxArticles` (type: `integer`):

Maximum articles for each query, topic or link (for a date range: for each day). Google News gives up to about 100. <code>0</code> = all.

## `decodeUrls` (type: `boolean`):

Turn Google News redirect links into the real publisher URLs (recommended). Articles whose link cannot be decoded are included for free.

## `fetchDetails` (type: `boolean`):

Open each article on the publisher's site and add the full text (clean paragraphs, ready for AI), main image, description, author and word count. Charged only when the full text is found.

## `onlyNew` (type: `boolean`):

For scheduled runs: skip articles already delivered by previous runs with the same search, country and language. Skipped articles are not charged. The list is kept in the key-value store <code>google-news-seen-articles</code> of your account.

## `proxyConfiguration` (type: `object`):

Not needed: the Actor connects directly and switches to Apify Proxy by itself if Google slows it down.

## Actor input object example

```json
{
  "queries": [
    "artificial intelligence",
    "site:reuters.com oil"
  ],
  "country": "US",
  "language": "en",
  "timeRange": "any",
  "maxArticles": 20,
  "decodeUrls": true,
  "fetchDetails": false,
  "onlyNew": false
}
```

# Actor output Schema

## `results` (type: `string`):

One row per article: title, source, date, direct publisher URL, related coverage and, if requested, full text, image and author.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ],
    "maxArticles": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("brii3343/google-news-scraper-full-text").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["artificial intelligence"],
    "maxArticles": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("brii3343/google-news-scraper-full-text").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ],
  "maxArticles": 20
}' |
apify call brii3343/google-news-scraper-full-text --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,brii3343/google-news-scraper-full-text"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7lKY6YmU8WNaSTjlH/builds/1zd373axUmTTuBQJk/openapi.json