# Google News Scraper - Real URLs & Article Text (`sauliusautomatesit/google-news-scraper`) Actor

Scrape Google News by keyword, topic, or headlines. Decodes Google's redirect links to real publisher URLs, with optional full article text and deduplicated monitoring.

- **URL**: https://apify.com/sauliusautomatesit/google-news-scraper.md
- **Developed by:** [Saulius Saulenas](https://apify.com/sauliusautomatesit) (community)
- **Categories:** News, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.55 / 1,000 articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google News Scraper — Real Article URLs + Full Text

Scrape Google News by keyword, topic, or top headlines — and get the **real publisher URL** for
every article (decoded from Google's redirect links), with **optional full article text**. Built for
news monitoring, competitor/brand intelligence, and AI/RAG pipelines.

No API key, no login.

### Why this scraper

- **Real article URLs, not Google redirects** — most Google News scrapers hand you an opaque
  `news.google.com/rss/articles/CB…` link. This actor decodes it to the actual publisher URL
  (e.g. `https://www.bloomberg.com/news/…`), so your links are usable everywhere.
- **Full article text** (optional) — extract readable body text, author, and image for each
  article. Best-effort by design: paywalled/bot-walled pages keep their metadata, note the reason,
  and are **not charged** the content fee.
- **Built for monitoring** — deduplicate across scheduled runs via a named store, so a daily run
  returns only what's new. Already-seen articles are skipped without wasting requests.
- **Search, topics & headlines** — keyword queries (with Google News operators), the 8 topic
  sections, and Top Headlines, in any language/country edition, with time-range filters.

### Input

```json
{
    "queries": ["\"electric vehicles\"", "openai"],
    "topics": ["technology", "business"],
    "includeTopHeadlines": false,
    "language": "en-US",
    "country": "US",
    "timeRange": "week",
    "maxArticlesPerQuery": 100,
    "resolveUrls": true,
    "extractArticleText": true,
    "dedupByUrl": true
}
```

| Field | Default | Description |
|---|---|---|
| `queries` | — | Search terms (support `"exact"`, `intitle:`, `site:`) |
| `topics` | `[]` | world, nation, business, technology, entertainment, sports, science, health |
| `includeTopHeadlines` | `false` | Also fetch the Top Headlines feed |
| `language` / `country` | `en-US` / `US` | News edition |
| `timeRange` | `any` | any, hour, day, week, month, year (search queries) |
| `maxArticlesPerQuery` | `100` | Cap per feed (~100 available) |
| `resolveUrls` | `true` | Decode Google links to real publisher URLs |
| `extractArticleText` | `false` | Also fetch and extract article body text |
| `dedupByUrl` | `true` | Skip articles seen in previous runs |
| `dedupStoreName` | `google-news-dedup` | Named store per watchlist |

Provide `queries`, `topics`, or enable `includeTopHeadlines`.

### Output

```json
{
    "title": "OpenAI completes $7 billion employee tender - Bloomberg",
    "source": "Bloomberg",
    "url": "https://www.bloomberg.com/news/articles/2026-08-10/openai-buys-back-7-billion",
    "googleNewsUrl": "https://news.google.com/rss/articles/CBMi…",
    "publishedAt": "2026-08-10T14:03:00.000Z",
    "query": "search:openai",
    "language": "en-US",
    "country": "US",
    "snippet": "OpenAI has completed a share sale…",
    "articleText": "…full readable body when extraction is enabled…",
    "wordCount": 812,
    "author": "Jane Doe",
    "image": "https://…/cover.jpg",
    "scrapedAt": "2026-08-11T23:10:00.000Z"
}
```

When `extractArticleText` is off, `articleText`/`wordCount`/`author` are omitted. When a page can't be
extracted, `articleText` is `null` and `extractionError` explains why (that row is not charged the
content fee).

### Pricing

Pay per event:

- **Article — $0.003** each ($3.00 / 1,000): one news article with metadata and the resolved real URL.
- **Article content — $0.002** each: charged **only** when full text is successfully extracted
  (`extractArticleText` on). Paywalled/blocked pages are free.
- **Actor start — $0.00005** per GB.

Platform usage (compute, proxy, storage) is **included**. Set **Maximum total charge** to cap spend.

| Run | You pay |
|---|---|
| 100 headlines, URLs resolved, no text | $0.30 |
| 100 articles with full text (70% extracted) | ~$0.44 |
| Daily monitor, ~20 new articles/day with text | ~$0.10/day |

### Use cases

- **Brand & PR monitoring** — schedule keyword runs; get only new coverage with real links.
- **Competitor intelligence** — track competitor mentions across sources.
- **Market research** — topic feeds by country/language for trend analysis.
- **AI / RAG** — feed resolved URLs and clean article text into LLM pipelines.
- **Newsletters & aggregators** — assemble fresh, deduplicated story lists.

### Limitations

- Google News returns up to ~100 items per feed; use multiple queries/topics or countries for breadth.
- Full-text extraction is best-effort — some publishers paywall or block automated fetches. Those
  rows still include the resolved URL and metadata, and aren't charged the content fee.
- Cross-run dedup uses a named store; use a distinct `dedupStoreName` per separate watchlist so
  their histories don't mix.

### FAQ

**Do I get the real article link?** Yes — that's the point. Keep `resolveUrls` on (default).

**Why isn't every article's text extracted?** Paywalls and anti-bot protection. You're never charged
the content fee for a failed extraction.

**How do I run a daily monitor?** Put your query in `queries`, keep `dedupByUrl` on, and schedule the
actor. Each run returns only new articles.

# Actor input Schema

## `queries` (type: `array`):

Keywords or phrases to search Google News for. Supports Google News operators (e.g. `"exact phrase"`, `intitle:`, `site:reuters.com`).

## `topics` (type: `array`):

Google News topic sections to pull. Valid: world, nation, business, technology, entertainment, sports, science, health.

## `includeTopHeadlines` (type: `boolean`):

Also fetch the main Top Headlines feed for the chosen country/language.

## `language` (type: `string`):

Interface/content language, e.g. `en-US`, `en-GB`, `de`, `fr`, `es-419`.

## `country` (type: `string`):

Two-letter country/edition code, e.g. `US`, `GB`, `DE`, `IN`.

## `timeRange` (type: `string`):

Restrict search-query results to a recent window. Topics/headlines are always current.

## `maxArticlesPerQuery` (type: `integer`):

Cap on articles taken from each query/topic/headlines feed (Google News returns up to ~100 per feed).

## `resolveUrls` (type: `boolean`):

Decode Google's redirect links into the real publisher URL (recommended). Disable for faster/cheaper runs that only need the Google News link.

## `extractArticleText` (type: `boolean`):

Also fetch each article and extract readable body text, author, and image. Best-effort — some publishers block or paywall content (those rows keep metadata and note the reason, and are not charged the content fee).

## `dedupByUrl` (type: `boolean`):

Skip articles already seen in previous runs (great for scheduled monitoring). Dedup key is the resolved URL when available.

## `dedupStoreName` (type: `string`):

Named storage used to remember seen articles across runs. Use one distinct name per monitoring watchlist so their histories don't mix.

## `concurrency` (type: `integer`):

Articles processed in parallel.

## `proxyConfiguration` (type: `object`):

Proxy settings. Apify Proxy (automatic) is recommended, especially with article-text extraction.

## Actor input object example

```json
{
  "queries": [
    "artificial intelligence"
  ],
  "includeTopHeadlines": false,
  "language": "en-US",
  "country": "US",
  "timeRange": "any",
  "maxArticlesPerQuery": 100,
  "resolveUrls": true,
  "extractArticleText": false,
  "dedupByUrl": true,
  "dedupStoreName": "google-news-dedup",
  "concurrency": 8,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `articles` (type: `string`):

One item per article: headline, publisher name, the **real publisher URL** decoded from Google's redirect link, the original Google News URL, publication date, the query or topic that produced it, language and country, and a short snippet. When full-text extraction is enabled, items also carry the article text, word count, author and lead image — or an `extractionError` when the publisher blocked extraction.

## `summary` (type: `string`):

A JSON report of the run: how many feeds were fetched, how many articles were returned, how many Google redirect links were successfully decoded to real publisher URLs, how many articles had their full text extracted, how many duplicates were skipped by cross-run deduplication, and whether the run stopped early at your maximum charge limit.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("sauliusautomatesit/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["artificial intelligence"] }

# Run the Actor and wait for it to finish
run = client.actor("sauliusautomatesit/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ]
}' |
apify call sauliusautomatesit/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,sauliusautomatesit/google-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gDyafe3UsIoTRBYEn/builds/jhtjFZfytWqdaIQ2N/openapi.json