# Google News Scraper — Decoded Article URLs, Any Edition (`scrapersdelight/google-news-scraper`) Actor

Scrape Google News search, topic and top-stories feeds into rows: headline, publisher, publish date, related coverage — and the REAL article URL, decoded out of Google's redirector instead of the useless news.google.com link. 100+ editions. No API key.

- **URL**: https://apify.com/scrapersdelight/google-news-scraper.md
- **Developed by:** [Scrapers Delight](https://apify.com/scrapersdelight) (community)
- **Categories:** News, Marketing, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.50 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 📰 Google News Scraper — with the REAL article URLs, not Google's redirector

Scrape **Google News** search results, topic sections and top stories into clean rows — and get the
**publisher's actual article URL** on every row, decoded out of the
`news.google.com/rss/articles/CBMi…` redirector that every other Google News feed hands you.

No API key. No login. No browser. Works on **every Google News edition** (US, UK, India, Germany,
Brazil, Japan…).

***

### 🔗 Why the decoded URL is the whole point

A raw Google News feed gives you links like:

```
https://news.google.com/rss/articles/CBMiaEFVX3lxTE9mQWFjYmw4NHEyTFlGWjE4UTBmdFhISmIwMlk5N0d3…
```

That link is useless downstream. You cannot dedupe by domain, you cannot fetch the full text, you
cannot check it against a paywall list, you cannot hand it to a client. Following it does not help
either — it serves a 592 KB JavaScript shim whose `rel=canonical` points back at itself.

This actor resolves it to the real thing:

```
https://www.cnbc.com/2026/09/01/ai-token-prices-lows.html
```

**Measured on a live validation run of 442 articles: 442/442 URLs decoded (100%)**, and 442/442 of
the decoded hostnames matched the publisher hostname the feed itself declared — so the decode is
verified against an independent signal, not just "it returned a string".

URL decoding is **included in the per-article price**. It is not an add-on event.

***

### ✅ What you get on every row

- 📰 **`title`** — the headline, with the `" - Publisher"` suffix stripped off
- 🏷️ **`source`** + **`sourcePublisherUrl`** — the outlet's name and home page
- 🔗 **`articleUrl`** — **the decoded publisher URL** (the reason this actor exists)
- 🧭 **`googleNewsUrl`** + **`articleId`** — the original link and Google's stable article id
- 🕒 **`publishedAt`** — ISO 8601 timestamp (plus the raw RFC-822 string)
- 🗞️ **`relatedArticles`** — other outlets covering the same story, from Google's own cluster
- 🌍 **`country`** / **`language`** — which edition the row came from
- 🧾 **`feedType`** / **`feedQuery`** / **`feedSection`** / **`positionInFeed`** — full provenance
- ✔️ **`urlDecoded`** / **`decodeStatus`** — never guess whether a link is real

#### Measured field fill (442 articles, 8 feeds, 2 editions, 2026-09-02)

| Field | Fill |
|---|---|
| `articleId`, `title`, `titleWithSource`, `source`, `sourcePublisherUrl` | **100%** |
| `articleUrl`, `urlDecoded`, `decodeStatus`, `googleNewsUrl` | **100%** |
| `publishedAt`, `publishedRaw` | **100%** |
| `feedType`, `feedUrl`, `country`, `language`, `positionInFeed`, `scrapedAt` | **100%** |
| `relatedArticles` | **36.9%** (163 rows, 610 related links) |

`relatedArticles` is not missing data — Google only builds a coverage cluster on its **section /
topic / top-stories** feeds. On the run above, related coverage appeared on 97 section rows and 66
top-stories rows, and on 0 keyword-search rows, because keyword search feeds do not carry one.

**Honest note:** Google News RSS carries **no article summary**. The `<description>` element only
repeats the headline and the publisher name. Rather than ship a "snippet" field that just duplicates
the title, this actor ships the structured related-coverage cluster instead.

***

### 🚀 How to use it

#### A daily brief with no keywords

```json
{ "sections": ["TOP", "BUSINESS", "TECHNOLOGY"], "countries": ["US"], "maxItems": 150 }
```

#### Media monitoring for a brand, past 24 hours

```json
{
  "queries": ["\"Acme Corp\"", "Acme recall"],
  "when": "1d",
  "excludeSites": ["yahoo.com"],
  "maxItems": 200
}
```

#### The same story across five editions

```json
{
  "queries": ["interest rates"],
  "countries": ["US", "GB", "IN", "CA", "AU"],
  "language": "en-US",
  "maxItems": 500
}
```

#### Just paste a Google News URL

```json
{ "newsUrls": ["https://news.google.com/topics/CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB"] }
```

Search, `/topics/…`, `/headlines/section/…`, the home page and existing `/rss/…` URLs are all
accepted and converted to the right feed automatically.

#### Run it with nothing at all

An empty input returns **Google News top stories** for the US edition. It never errors out with
"nothing to scrape".

***

### 🔎 Google News search operators all work

Anything you can type into Google News search goes straight through `queries`:

| Operator | Example | Measured result |
|---|---|---|
| Exact phrase | `"electric vehicle" tesla` | 100 articles |
| OR | `tesla OR rivian` | 100 articles |
| Single site | `site:reuters.com tesla` | 100 articles |
| Multiple sites | `(site:reuters.com OR site:bbc.co.uk) tesla` | 100 articles |
| Exclude a site | `tesla -site:yahoo.com` | 100 articles |
| Headline only | `intitle:tesla` | 100 articles |
| Time window | `tesla when:1h` | 6 articles |

`includeSites`, `excludeSites` and `when` build these for you if you would rather not write them by
hand.

***

### 📈 Getting more than 100 articles

Google News RSS returns **at most ~100 articles per feed and has no page parameter.** No scraper can
change that — anyone claiming deep pagination on a single query is not reading the same feed.

What actually works, and what this actor is built for, is **fan-out**:

- **more queries** — each is its own feed
- **more countries** — `["US","GB","IN","CA","AU"]` runs every query once per edition
- **narrower time windows** — `when: "1d"` run daily beats one `when: "7d"` run weekly
- **`site:` slices** — split one broad query into one query per publisher

Measured: 8 feeds across 2 editions returned **322 unique articles** in a single run.

***

### 💰 Pricing

| | |
|---|---|
| Model | **Pay per event** |
| `article-scraped` | **$0.0015 per article** |
| Decoded publisher URL | **included** |
| Related coverage | **included** |
| Actor start | **not charged** |

You are charged for articles **delivered to your dataset**, and the charge happens as each batch is
pushed — so if you set a spend limit, the run stops cleanly at it instead of handing you rows you
already paid for. Duplicate articles across feeds are removed **before** they are charged.

1,000 articles ≈ **$1.50**.

***

### ⚙️ Input reference

| Field | Type | Default | What it does |
|---|---|---|---|
| `queries` | array | `[]` | Keywords to search. Supports Google News operators. |
| `sections` | array | `[]` | `TOP`, `WORLD`, `NATION`, `BUSINESS`, `TECHNOLOGY`, `ENTERTAINMENT`, `SPORTS`, `SCIENCE`, `HEALTH`. |
| `newsUrls` | array | `[]` | Any `news.google.com` URL, converted to its feed. |
| `language` | string | `en-US` | Google News `hl` value. |
| `countries` | array | `["US"]` | Google News `gl` values — one run per country. |
| `when` | enum | `any` | `1h`, `12h`, `1d`, `7d`, `30d`, `1y`. Applies to `queries`. |
| `includeSites` | array | `[]` | Restrict queries to these publisher domains. |
| `excludeSites` | array | `[]` | Drop these publisher domains from queries. |
| `decodeArticleUrls` | boolean | `true` | Resolve the real publisher URL. |
| `includeRelatedArticles` | boolean | `true` | Ship the coverage cluster. |
| `maxItems` | integer | `200` | Total cap. `0` = unlimited. |
| `maxItemsPerFeed` | integer | `100` | Per-feed cap. |
| `proxyConfiguration` | object | no proxy | Optional Apify Proxy. |

***

### ❓ FAQ

**1. Do I need a Google API key or a Google News account?**
No. This reads Google News' own public RSS feeds. There is no key, no login and no quota to apply for.

**2. What exactly is `articleUrl` and how is it different from `googleNewsUrl`?**
`googleNewsUrl` is Google's redirector (`news.google.com/rss/articles/CBMi…`). `articleUrl` is the
publisher's own URL that redirector points to — `https://www.reuters.com/...`. Only the second one is
usable downstream.

**3. Is the URL decoding reliable?**
On the validation run it resolved **442 of 442** articles, and every decoded hostname matched the
publisher hostname declared in the feed. It is not guaranteed forever — it depends on a Google
endpoint that Google can change. If it ever fails, rows still ship with the headline, publisher, date
and `googleNewsUrl`, and `decodeStatus` says `failed`. **The run does not fail and you still get your
data.**

**4. Can I get the full article text?**
Not from this actor — it is a Google News feed scraper, and one actor should do one job. Because you
get the real `articleUrl`, you can pipe the output straight into any article-extractor actor.

**5. Why do I only get ~100 results for my keyword?**
That is Google News RSS' own hard cap, not a limit of this actor. Fan out on `countries`, `when`, or
`site:` slices — see *Getting more than 100 articles* above.

**6. Why is `relatedArticles` empty on my rows?**
You are almost certainly using `queries`. Google only builds coverage clusters on its section, topic
and top-stories feeds. Add `"sections": ["TOP"]` to see them.

**7. Which countries and languages work?**
Every Google News edition. US, GB, IN, FR, ES, BR and JP were all verified returning 100 articles per
search feed. Set `language` (`hl`) and `countries` (`gl`) to match the edition you want.

**8. Can I monitor a single publisher?**
Yes — `"includeSites": ["reuters.com"]`, or write `site:reuters.com` directly in the query.

**9. How do I run this on a schedule?**
Use Apify Schedules. A common pattern is `when: "1d"` once a day, or `when: "1h"` hourly for
breaking-news monitoring. Articles are deduplicated inside a run by Google's stable article id.

**10. Do I need a proxy?**
No. Google News served Apify's egress directly in testing, so the default is no proxy. Turn on Apify
Proxy (RESIDENTIAL) if you run at high volume or want a country's edition served from inside that
country.

**11. Are duplicate articles charged twice?**
No. The same article appearing in several feeds is removed before anything is pushed or charged.

**12. What happens if Google blocks the run?**
Blocked requests are counted and named in the run's status message and log. A run that could not
reach Google at all **fails loudly** rather than handing you an empty dataset that looks like "there
was no news".

***

### ⚖️ Legal and fair use

This actor reads **publicly available Google News RSS feeds** — the same feeds any RSS reader
subscribes to — and resolves the redirect links they contain. It collects **no personal data**, uses
no login and no credentials.

Output is headline metadata and links: headlines, publisher names, timestamps and URLs. It does not
copy article bodies. Headlines and publisher names remain the property of their publishers; if you
republish them, respect the publishers' terms and applicable copyright law, and link back to the
original article — which is exactly what `articleUrl` is for.

You are responsible for how you use the data, including compliance with Google's terms and any
applicable law in your jurisdiction. Keep request volumes reasonable.

# Actor input Schema

## `queries` (type: `array`):

Keywords to search Google News for — one feed per query, up to ~100 articles each. Google News search operators work here: `"exact phrase"`, `tesla OR rivian`, `intitle:layoffs`, `site:reuters.com`. Leave every input empty and the actor returns Google News top stories instead.

## `sections` (type: `array`):

Google News' own front-page sections. `TOP` is the main top-stories feed; the rest are the standard topic sections. These are broad, always-populated feeds — good for a daily news sweep with no keyword.

## `newsUrls` (type: `array`):

Paste any news.google.com URL straight from your browser — a search page, a `/topics/<id>` page, a `/headlines/section/...` page, the home page, or an existing `/rss/...` feed. Each is converted to its RSS feed automatically.

## `language` (type: `string`):

Google News interface/content language, e.g. `en-US`, `en-GB`, `fr`, `de`, `es`, `pt-BR`, `ja`. Combined with each country below to select the edition.

## `countries` (type: `array`):

Two-letter country codes for the Google News editions to read, e.g. `US`, `GB`, `IN`, `CA`, `AU`, `DE`, `FR`, `BR`, `JP`. Every query and section is run once per country, which is how you get past the ~100-article cap on a single feed.

## `when` (type: `string`):

Restrict search queries to articles published inside this window (Google's own `when:` operator). Applies to `queries` only — sections and pasted URLs are always Google's current feed. Note `1h` can legitimately return only a handful of articles.

## `includeSites` (type: `array`):

Restrict search queries to these publisher domains, e.g. `reuters.com`, `bbc.co.uk`. Multiple domains are OR-ed together. Applies to `queries` only.

## `excludeSites` (type: `array`):

Drop these publisher domains from search queries, e.g. `yahoo.com`. Useful for stripping syndication mirrors out of a media-monitoring feed. Applies to `queries` only.

## `decodeArticleUrls` (type: `boolean`):

Resolve each `news.google.com/rss/articles/CBMi...` redirector into the publisher's own URL (e.g. `https://www.reuters.com/...`). This is what makes the rows usable for anything downstream — link checking, full-text fetching, dedupe by domain. Included in the per-article price; switch it off only if you want the fastest possible run and do not need the links.

## `includeRelatedArticles` (type: `boolean`):

On Google's cluster feeds (top stories, sections, topics) each story carries a list of other outlets covering the same event. Ship that list with the row as `relatedArticles`. Search feeds rarely carry one.

## `maxItems` (type: `integer`):

Stop after this many articles across all feeds. The default is deliberately small so a first run is cheap — raise it once you know the scope you want. 0 = no limit.

## `maxItemsPerFeed` (type: `integer`):

Cap each individual feed. Google News RSS itself returns at most ~100 articles per feed and has no page parameter, so values above 100 have no effect.

## `proxyConfiguration` (type: `object`):

Optional. Google News RSS served Apify's egress without a proxy in testing, so the default is no proxy. Turn on Apify Proxy (RESIDENTIAL) if you run at high volume, need a specific country's edition served from that country, or start seeing blocked-request warnings in the log.

## Actor input object example

```json
{
  "queries": [
    "artificial intelligence"
  ],
  "sections": [],
  "newsUrls": [],
  "language": "en-US",
  "countries": [
    "US"
  ],
  "when": "any",
  "includeSites": [],
  "excludeSites": [],
  "decodeArticleUrls": true,
  "includeRelatedArticles": true,
  "maxItems": 20,
  "maxItemsPerFeed": 100,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `articles` (type: `string`):

The dataset of scraped Google News articles (one item per article).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ],
    "language": "en-US",
    "decodeArticleUrls": true,
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapersdelight/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["artificial intelligence"],
    "language": "en-US",
    "decodeArticleUrls": True,
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("scrapersdelight/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ],
  "language": "en-US",
  "decodeArticleUrls": true,
  "maxItems": 20
}' |
apify call scrapersdelight/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapersdelight/google-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0kUWTcNqBGC2uLDAd/builds/ysDAxWwE8WTqccMcX/openapi.json