# Google News Scraper — Search & Topic RSS Feeds (`devilscrapes/google-news-scraper`) Actor

Scrape Google News search results and topic (beat) feeds into a clean dataset — title, publisher, source URL, publish date, and an opt-in true-article-URL resolver. Handles the ~100-item pagination cap and Google's redirect-token links so you don't have to.

- **URL**: https://apify.com/devilscrapes/google-news-scraper.md
- **Developed by:** [DevilScrapes](https://apify.com/devilscrapes) (community)
- **Categories:** News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

<div align="center">
  <img src=".actor/icon.svg" width="160" alt="Devil Scrapes mark" />

## Google News Scraper — Search & Topic RSS Feeds

**💰 $1.20 / 1 000 results**  ·  pay only for results  ·  no credit card to try

*We do the dirty work so your dataset stays clean.* 😈

Scrape Google News search results and topic (beat) feeds into a clean, deduped dataset — title, publisher, source domain, publish date, and an opt-in true-article-URL resolver. We walk the date-window pagination past Google's ~100-item cap and unwrap the redirect-token link so you get structured rows, not raw XML.

</div>

***

### 🎯 What this scrapes

Google News hands back a redirect token instead of the real article URL and quietly truncates any window at 100 items — we walk the date range and hand you clean rows anyway. This isn't a job for our generic **rss-feed-scraper** (that one parses any RSS/Atom feed as-is) — this Actor is purpose-built for Google News' redirect tokens, truncation-aware date-window pagination, and topic/beat feeds.

### 🔥 What we handle for you

- 🛡️ **Browser fingerprint rotation** — `curl-cffi` impersonates real Chrome / Firefox / Safari TLS handshakes so the target sees a browser, not Python.
- 🌐 **No proxy markup** — Google's RSS surface doesn't gate on IP reputation, so we don't burn your run budget on residential-proxy rotation you don't need; `proxyConfiguration` stays in the input in case that ever changes.
- 🔁 **Retries with exponential backoff** on `408 / 429 / 5xx` — up to 5 attempts per page, `Retry-After` honoured.
- 🧱 **Rate-limit-aware pacing** — when the target pushes back, we slow down instead of getting banned.
- 🧊 **Clean, typed dataset rows** — Pydantic-validated, ISO-8601 timestamps, stable IDs, JSON / CSV / Excel export straight from the Apify Console.
- 💰 **Pay-Per-Event pricing** — you only pay for results that hit your dataset. No data, no charge.

### 💡 Use cases

- PR and media-monitoring teams tracking brand or competitor coverage across outlets.
- Market researchers measuring news volume and sentiment shifts around a topic over time.
- SEO and content-ops teams tracking which publishers are covering a competitor or keyword.

### ⚙️ How to use it

1. Click **Try for free** at the top of the page.
2. Fill in the input form — most fields have sensible defaults.
3. Click **Start**. Output streams into the run's dataset.
4. Export from **Storage → Dataset** as JSON, CSV, or Excel — or fetch via the API.

### 📥 Input

| Field | Type | Required | Default | Notes |
|---|---|:--:|---|---|
| `queries` | `array` | no | \['openai', 'anthropic'] | Search queries. Each is walked independently across the resolved date range. At least one of Queries / Topics is… |
| `topics` | `array` | no | \[] | Google News topic (beat) codes, e.g. TECHNOLOGY, BUSINESS. At least one of Queries / Topics is required. |
| `hl` | `string` | no | 'en-US' | Google News language parameter. |
| `gl` | `string` | no | 'US' | Google News country parameter, 2-letter. |
| `ceid` | `string` | no | 'US:en' | Combined locale parameter, shape "{gl}:{lang}". |
| `dateFrom` | `string` | no | '—' | Start of the date walk (Queries mode only). Defaults to 7 days before run start when both dates are omitted. |
| `dateTo` | `string` | no | '—' | End of the date walk (Queries mode only). Defaults to run start when omitted. |
| `resolveArticleUrl` | `boolean` | no | False | Follow each item's Google redirect link to populate the real publisher article URL. Costs one extra request per item —… |
| `maxItemsPerQuery` | `integer` | no | 500 | Cap on emitted rows per query or topic. |
| `proxyConfiguration` | `object` | no | {'useApifyProxy': False} | Apify Proxy configuration. This target is proven reachable proxy-free — not defaulted to RESIDENTIAL. |

#### Example input

```json
{
  "queries": [
    "openai",
    "anthropic"
  ],
  "dateFrom": "2026-09-11",
  "dateTo": "2026-09-13",
  "resolveArticleUrl": false,
  "maxItemsPerQuery": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

### 📤 Output

Every row is one dataset item.

| Field | Type | Notes |
|---|---|---|
| `title` | `string` | Article title, kept exactly as received (including any trailing " - Publisher" suffix). |
| `link` | `string` | Raw Google News redirect token URL. |
| `resolved_url` | `['string', 'null']` | True publisher article URL — only populated when resolveArticleUrl=true. |
| `source_name` | `['string', 'null']` | Publisher name from <source>. |
| `source_url` | `['string', 'null']` | Publisher domain from <source url="...">. |
| `pub_date` | `string` | Publish date/time, ISO-8601. |
| `query` | `['string', 'null']` | Input query that produced this row (queries mode). |
| `topic` | `['string', 'null']` | Input topic code that produced this row (topics mode). |
| `guid` | `string` | Feed GUID — primary dedupe key. |

#### Example output

```json
{
  "title": "OpenAI announces new model - TechCrunch",
  "link": "https://news.google.com/rss/articles/CBMi...",
  "resolved_url": null,
  "source_name": "TechCrunch",
  "source_url": "https://techcrunch.com",
  "pub_date": "2026-09-10T14:32:00Z",
  "query": "openai",
  "topic": null,
  "guid": "CBMi..."
}
```

### 💰 Pricing

Pay-Per-Event — you pay only when these events fire:

| Event | USD | What it is |
|---|---:|---|
| `actor-start` | $0.2 | One-off warm-up charge per run |
| `result` | $0.001 | Per dataset item |

Example: 1 000 results at the rates above ≈ **$1.20**. No subscription, no minimum, no card to start — Apify gives every new account $5 of free credit.

### 🚧 Limitations

resolveArticleUrl defaults off — turning it on adds one request per row, so unit economics favor leaving it off unless you need the resolved publisher URL. The ~100-item-per-window cap is handled by halving down to a 1-day floor; extremely high-volume queries can still hit 100 items even at that floor, in which case we accept the window best-effort and log a warning rather than guess at what's missing. Topic feeds return a snapshot with no date window. This is an undocumented Google endpoint with no published SLA — we watch for shape changes, but Google can alter it without notice.

### ❓ FAQ

**Why is resolveArticleUrl off by default?**

Turning it on follows one extra redirect per row, which multiplies request volume. We keep it opt-in so the default run stays fast and cheap — flip it on when you need the true publisher URL instead of Google's redirect token.

**What happens when a date window has more than 100 articles?**

Google truncates any window at exactly 100 items without saying so. We detect that ceiling, split the window in half, and re-fetch — recursively, down to a 1-day floor — so dense windows still surface everything reachable instead of silently dropping items.

**Does this work for topic feeds like TECHNOLOGY or BUSINESS?**

Yes — pass topic codes instead of (or alongside) search queries to monitor a beat. Topic feeds return Google's current snapshot and aren't date-windowed, since Google doesn't expose a date cursor for them.

**Where does source\_url come from if link is a redirect token?**

Google's own <source> element carries the publisher's domain for free — no extra request needed. We extract it directly, so you get a usable publisher domain on every row even with resolveArticleUrl off.

### 💬 Your feedback

Spotted a bug, hit a weird edge case, or need a new field? Open an
issue on the Actor's **Issues** tab on Apify Console — we ship
fixes weekly and we read every report.

***

<div align="center">

Built by **[Devil Scrapes](https://apify.com/DevilScrapes)** 😈 — a small fleet of
opinionated public-data Actors. Honest pricing, real engineering, zero fine print.

</div>

# Actor input Schema

## `queries` (type: `array`):

Search queries. Each is walked independently across the resolved date range. At least one of Queries / Topics is required.

## `topics` (type: `array`):

Google News topic (beat) codes, e.g. TECHNOLOGY, BUSINESS. Monitors a beat instead of a search query — no date windowing applies. At least one of Queries / Topics is required.

## `hl` (type: `string`):

Google News language parameter.

## `gl` (type: `string`):

Google News country parameter, 2-letter.

## `ceid` (type: `string`):

Combined locale parameter, shape "{gl}:{lang}".

## `dateFrom` (type: `string`):

Start of the date walk (Queries mode only). If both Date from / Date to are omitted, defaults to 7 days before run start.

## `dateTo` (type: `string`):

End of the date walk (Queries mode only). If omitted, defaults to run start.

## `resolveArticleUrl` (type: `boolean`):

Follow each item's Google redirect link to populate the real publisher article URL. Costs one extra request per item — off by default.

## `maxItemsPerQuery` (type: `integer`):

Cap on emitted rows per query or topic before the Actor stops walking further windows/pages for it.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration. This target is proven reachable proxy-free — do not default to RESIDENTIAL.

## Actor input object example

```json
{
  "queries": [
    "openai",
    "anthropic"
  ],
  "topics": [],
  "hl": "en-US",
  "gl": "US",
  "ceid": "US:en",
  "dateFrom": "2026-09-01",
  "dateTo": "2026-09-13",
  "resolveArticleUrl": false,
  "maxItemsPerQuery": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetItems` (type: `string`):

No description

## `datasetItemsCsv` (type: `string`):

No description

## `datasetView` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "openai",
        "anthropic"
    ],
    "topics": [],
    "hl": "en-US",
    "gl": "US",
    "ceid": "US:en",
    "dateFrom": "2026-09-01",
    "dateTo": "2026-09-13",
    "resolveArticleUrl": false,
    "maxItemsPerQuery": 50,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("devilscrapes/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "openai",
        "anthropic",
    ],
    "topics": [],
    "hl": "en-US",
    "gl": "US",
    "ceid": "US:en",
    "dateFrom": "2026-09-01",
    "dateTo": "2026-09-13",
    "resolveArticleUrl": False,
    "maxItemsPerQuery": 50,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("devilscrapes/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "openai",
    "anthropic"
  ],
  "topics": [],
  "hl": "en-US",
  "gl": "US",
  "ceid": "US:en",
  "dateFrom": "2026-09-01",
  "dateTo": "2026-09-13",
  "resolveArticleUrl": false,
  "maxItemsPerQuery": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call devilscrapes/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,devilscrapes/google-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/d0i2iDwK1kM8qsduq/builds/bCevwcNU6BWgIcEA7/openapi.json