# Google News Scraper (`excellent_mustang/google-news-scraper`) Actor

- **URL**: https://apify.com/excellent\_mustang/google-news-scraper.md
- **Developed by:** [Gorav Agarwal](https://apify.com/excellent_mustang) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google News Scraper

Scrape Google News at scale and get back **the real publisher URL for every article** — not the opaque `news.google.com/rss/articles/CBMi...` redirect that most Google News scrapers leave you with. Search by keyword, browse a topic section, filter by publisher or date range, and run the same query across 50+ country editions and 40 languages in a single run.

No API key. No login. No browser. Typically **under 15 seconds for 100 fully-resolved articles**.

### What this Google News scraper does

This Actor turns Google News into a clean, structured dataset you can query, diff and monitor. Point it at a keyword and it returns every matching article Google is currently surfacing, with the headline, publisher, publication timestamp, direct article link, and Google's own story-clustering (which other outlets are covering the same story).

#### Why "resolve the real URL" matters

Every link in a Google News feed is an encoded redirect. Since Google retired the old encoding scheme, those identifiers no longer contain the destination — they are opaque protobuf handles. A scraper that simply returns them hands you data you cannot use: you cannot deduplicate by domain, you cannot tell a wire reprint from an original, you cannot feed the URL to a summariser, and the link rots.

This Actor calls Google's own internal resolution endpoint to recover the true publisher URL for each article — in **batches of 20 per request**, so resolution costs about one extra second per 20 articles rather than one page load each. In testing across US, UK, German and Japanese editions, resolution succeeded for **100% of articles**.

### How it works

1. **Build the feed URLs.** Your input is expanded into one RSS feed per (query × market) combination. Topic sections use a generated Google News topic token, so built-in sections like Business or Technology work in *every* locale — not just US English.
2. **Fetch feeds concurrently.** Each feed returns up to ~100 articles. Results are merged and de-duplicated by Google's article ID.
3. **Resolve publisher URLs.** Article signatures are harvested in parallel, then decoded in batched RPC calls.
4. **Optionally fetch thumbnails** from each publisher's own `og:image` tag.
5. **Push to the dataset**, honouring your run's charge limit.

The pipeline is built to degrade rather than fail. If URL resolution or image lookup times out, articles are still returned with their Google News link intact, and the run succeeds. A per-run enrichment time budget guarantees the Actor never runs long.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `query` | string | — | Keyword or phrase. Supports Google News operators: `OR`, `-exclude`, `"exact phrase"`, `intitle:`, `allintext:` |
| `topic` | enum | — | Built-in section: World, Nation, Business, Technology, Entertainment, Sports, Science, Health |
| `publisher` | string | — | Restrict to one domain, e.g. `reuters.com` |
| `language` | enum | `en` | 40 languages |
| `region` | enum | `US` | 56 country editions |
| `additionalMarkets` | array | `[]` | Extra editions as `REGION:language`, e.g. `["GB:en","DE:de","JP:ja"]` |
| `dateFrom` / `dateTo` | date | — | Publication window (`YYYY-MM-DD`) |
| `maxItems` | integer | `50` | Total articles across all feeds |
| `resolveArticleUrls` | boolean | `true` | Recover real publisher URLs |
| `includeImages` | boolean | `false` | Fetch article thumbnails |

Leave every field empty and the Actor scrapes the Top Stories front page for the selected market, so a bare run always returns data.

#### Example input

```json
{
  "query": "semiconductor export controls",
  "additionalMarkets": ["GB:en", "DE:de", "JP:ja"],
  "dateFrom": "2026-08-01",
  "maxItems": 200,
  "resolveArticleUrls": true
}
```

### Output example

One dataset item per article. Real output from a `TECHNOLOGY` topic run:

```json
{
  "position": 3,
  "title": "Five ways that the new Siri AI is frustrating iPhone and Apple Watch users",
  "description": "Five ways that the new Siri AI is frustrating iPhone and Apple Watch users",
  "url": "https://9to5mac.com/2026/09/20/five-ways-that-the-new-siri-ai-is-frustrating-iphone-and-apple-watch-users/",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMiqgFBVV95cUxNZUV2WFp0U0x5Tnl3cTJCQ3Q1N2htcXlxT0lvXzUzbmhsSHJoZEU3MHJ4V2Fy...?oc=5",
  "urlResolved": true,
  "source": "9to5Mac",
  "sourceUrl": "https://9to5mac.com",
  "sourceDomain": "9to5mac.com",
  "image": "https://9to5mac.com/wp-content/uploads/sites/6/2026/09/iphone-18-pro-ios-27-siri-ai.jpg",
  "publishedAt": "2026-09-20T18:35:00Z",
  "publishedAtRaw": "Sun, 20 Sep 2026 18:35:00 GMT",
  "relatedCoverage": [
    { "title": "The Top New Features in Apple's iOS 27 and iPadOS 27", "source": "WIRED", "googleNewsUrl": "https://news.google.com/rss/articles/CBMinwFBVV95cUxOald3VG43..." },
    { "title": "With iOS 27, Apple Is Laying the Groundwork to Paywall Your iPhone", "source": "pcmag.com", "googleNewsUrl": "https://news.google.com/rss/articles/CBMinAFBVV95cUxOeGUxR09n..." }
  ],
  "relatedCoverageCount": 4,
  "query": null,
  "topic": "TECHNOLOGY",
  "language": "en",
  "region": "US",
  "market": "US:en",
  "articleId": "CBMiqgFBVV95cUxNZUV2WFp0U0x5Tnl3cTJCQ3Q1N2htcXlxT0lvXzUz...",
  "scrapedAt": "2026-09-20T23:37:14Z"
}
```

`relatedCoverage` is Google's own story clustering — the other outlets running the same story. On topic and Top Stories feeds it is populated for the large majority of items, which makes it useful for measuring share of voice and spotting wire pickup.

### Pricing

This Actor uses **pay per result**. You are charged per article written to the dataset, and nothing else — no platform usage charges, no per-run fee beyond the standard Actor start event. A run that returns 40 articles costs 40 results.

Because articles are de-duplicated across markets before pushing, running one query against five country editions does not bill you five times for the same syndicated story.

### Use cases

**Media monitoring and PR.** Track mentions of a brand, executive or product across countries, and see which outlets picked up a story first via `relatedCoverage`.

**Competitive and market intelligence.** Watch competitor announcements, funding, regulatory action or recalls. The `publisher` filter plus a date window turns this into a targeted archive query.

**Building a news dataset for LLMs and RAG.** Resolved publisher URLs are directly fetchable, which is exactly what a downstream summarisation, embedding or classification pipeline needs. Google redirect links are not.

**Finance and event-driven research.** Schedule a run every 15 minutes on a ticker or commodity and diff the dataset to get a clean stream of new headlines with reliable timestamps.

**Google News API alternative.** Google has no official public News API. This Actor is a stable, structured, no-authentication substitute you can call from the Apify API, CLI, MCP, or any scheduler.

**Cross-market narrative comparison.** Run the same query in `US:en`, `DE:de` and `JP:ja` and compare which outlets and framings dominate each edition.

### FAQ & Support

**Do I need a Google API key or account?** No. The Actor uses no authentication of any kind.

**How many articles can I get per query?** Google News serves a maximum of roughly 100 items per feed, and that is a hard limit on Google's side — no scraper can page beyond it. To go further, widen the search: add markets via `additionalMarkets`, split a broad topic into several narrower queries, or slice a date range into windows. Each feed contributes up to ~100 more articles.

**How far back does the date filter reach?** Google honours `dateFrom`/`dateTo` for roughly the last 12 months on most queries. The filter is Google's own and is approximate at the edges — expect a small number of results just outside the window.

**Why is `description` often the same as `title`?** For single-article items Google News RSS does not provide a separate snippet; the feed's description field contains only the headline and source. This Actor reports that faithfully rather than inventing a summary. Cluster items carry real extra information in `relatedCoverage`.

**Why do some articles have no image?** Thumbnails are read from the publisher's own page, and some publishers block non-browser requests. Expect a thumbnail for roughly two thirds of articles when `includeImages` is on. Image lookup is off by default because it materially increases run time.

**Can it scrape the full article text?** No, by design. This Actor returns Google News metadata and the canonical article URL. Full-text extraction is a separate concern with its own copyright considerations — pipe the `url` field into a dedicated article-extraction Actor if you need body text.

**Does it need a proxy?** No. Google News RSS is reachable from Apify's datacenter IPs. Apify Proxy is available in the input for very large jobs but is off by default, and a proxy failure degrades to a direct connection rather than failing the run.

**Is scraping Google News legal?** This Actor reads only publicly available, non-personal Google News RSS feeds — no login, no personal data, no paywalled content. You are responsible for how you use the output, particularly in respect of publishers' copyright in headlines and article text.

**Something looks wrong.** Open an issue on the Actor's Issues tab with your input JSON and the run ID. Google occasionally changes the article-ID encoding; if resolution success drops, that is the cause and it will be fixed promptly.

# Actor input Schema

## `query` (type: `string`):

Keyword or phrase to search Google News for. Supports Google News operators, e.g. "tesla OR rivian", ""supply chain"", "intitle:earnings". Leave empty to use a Topic instead, or to scrape Top Stories.

## `topic` (type: `string`):

Scrape one of Google News' built-in headline sections instead of running a keyword search. Ignored when a Search query is set.

## `publisher` (type: `string`):

Only return articles from this domain, e.g. "reuters.com" or "bbc.co.uk". Applied as a site: filter on top of your query.

## `language` (type: `string`):

Article language (the Google News 'hl' parameter).

## `region` (type: `string`):

Country edition of Google News (the 'gl' parameter). Different editions surface different outlets for the same query.

## `additionalMarkets` (type: `array`):

Run the same query against extra editions in one go, as REGION:language pairs (e.g. "GB:en", "DE:de", "JP:ja"). Results are merged and de-duplicated. Ideal for tracking how a story is covered across countries.

## `dateFrom` (type: `string`):

Only return articles published on or after this date (YYYY-MM-DD). Google News honours this back to roughly the last 12 months for most queries.

## `dateTo` (type: `string`):

Only return articles published on or before this date (YYYY-MM-DD).

## `maxItems` (type: `integer`):

Total number of articles to return across all feeds. Google News serves up to ~100 items per feed, so add more markets to go beyond that.

## `resolveArticleUrls` (type: `boolean`):

Convert each opaque news.google.com/rss/articles/... link into the actual publisher URL (e.g. https://www.reuters.com/...). Strongly recommended — without it you only get Google redirect links. Adds roughly 10 seconds per 100 articles.

## `includeImages` (type: `boolean`):

Look up each article's og:image on the publisher's own page. Requires URL resolution. Typically finds a thumbnail for around two thirds of articles — some publishers block non-browser traffic — and noticeably increases run time.

## `maxConcurrency` (type: `integer`):

Parallel HTTP requests. The default is a good balance; raise it only if you are scraping thousands of articles and using a proxy.

## `enrichmentTimeoutSecs` (type: `integer`):

Hard ceiling on URL resolution and image lookup combined. When the budget runs out the Actor stops enriching and returns what it already has, rather than running long.

## `proxyConfiguration` (type: `object`):

Optional. Google News RSS is reachable without a proxy, so this is off by default. Enable Apify Proxy if you are running very large jobs or scraping from a blocked network.

## Actor input object example

```json
{
  "query": "artificial intelligence",
  "topic": "",
  "publisher": "reuters.com",
  "language": "en",
  "region": "US",
  "additionalMarkets": [],
  "dateFrom": "2026-01-01",
  "dateTo": "2026-02-01",
  "maxItems": 50,
  "resolveArticleUrls": true,
  "includeImages": false,
  "maxConcurrency": 10,
  "enrichmentTimeoutSecs": 150,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `articles` (type: `string`):

All news articles collected by this run, one item per article, with resolved publisher URLs.

## `articlesOverview` (type: `string`):

The same articles trimmed to the headline columns: title, source, published date and URL.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "artificial intelligence",
    "maxItems": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("excellent_mustang/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "artificial intelligence",
    "maxItems": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("excellent_mustang/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "artificial intelligence",
  "maxItems": 50
}' |
apify call excellent_mustang/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,excellent_mustang/google-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/EE3LBGtR64WupzrGc/builds/tb7wdGAf0f2LGUxDC/openapi.json