# Google News Scraper — Articles, Sources & Full Text (`datakiln/google-news-scraper`) Actor

Scrape Google News by keyword or topic in any language and country, with real publisher URLs (decoded), dates, sources, and optional full article text, author and image. Goes past the 100-result limit by splitting dates.

- **URL**: https://apify.com/datakiln/google-news-scraper.md
- **Developed by:** [DataKiln Labs](https://apify.com/datakiln) (community)
- **Categories:** News, SEO tools, AI
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google News Scraper — articles, sources & full text

Search Google News by keyword or pull topic headlines in **any language and country**, and get every article with its **real publisher URL** (not a `news.google.com` redirect), source, publish date and — optionally — the **full article text, author, top image and language** from the publisher's page. Built for media monitoring, brand and competitor tracking, PR reports, market research, news datasets and LLM/RAG pipelines.

**$1 per 1,000 articles ($0.001 each). Full text: +$0.002 per article, charged only when the text was actually extracted.** No login, no API key.

### Why this Google News scraper

- **Real article URLs.** Google News RSS links are encrypted redirects (`news.google.com/rss/articles/CBMi…`) that break in spreadsheets, CRMs and crawlers. Every link is resolved to the publisher's real URL; the few that can't be resolved are flagged with `decodeFailed: true` instead of silently returning junk.
- **More than 100 results per query.** Google returns at most ~100 articles per search. When that cap is hit, the scraper automatically splits your period into single days (`after:` / `before:`) and collects up to 100 per day — e.g. 7 days can give several hundred unique articles for a busy topic.
- **Full text in the same run.** Turn on *Extract full article text* to get the main text, author, publish date, top image and language, read from the article's structured data (JSON-LD `NewsArticle`) and a readability-style text extractor. You pay the extra only for articles where text was found.
- **Any edition, many at once.** English US/UK/India, Arabic (Saudi Arabia, Egypt, UAE…), German, French, Spanish, Portuguese (Brazil), Japanese and every other Google News edition — run several editions in one go, de-duplicated.
- **No duplicates.** Articles are de-duplicated by their real URL across queries, topics, day windows and editions.

### What you get

| Field | Description |
|---|---|
| `title` | Headline (the " - Publisher" suffix Google adds is removed) |
| `publisher`, `publisherUrl` | Source name and homepage, e.g. `Reuters`, `https://www.reuters.com` |
| `publishedAt` | Publish time in ISO 8601 (UTC) |
| `url` | **Real article URL** on the publisher's site |
| `decodeFailed` | `true` if the Google link couldn't be resolved (then `url` is the Google link) |
| `googleNewsUrl` | Original Google News link |
| `description` | Article description (from the publisher's page when full text is on) |
| `query` / `topic` | Which search query or topic found the article |
| `locale` | Edition, e.g. `en-US:US`, `ar:SA` |
| `position`, `dateWindow` | Rank in Google's feed and the day window it came from (`null` = main feed) |
| `relatedArticles` | For topic headlines: other outlets covering the same story (title, publisher, Google link) |
| `scrapedAt` | Timestamp |

**With full text on:** `articleTitle`, `author`, `articlePublishedAt`, `articleText`, `textLength`, `topImage`, `language`, `paywalled`, `fullTextStatus` (`ok`, `paywalled`, `blocked`, `no-text`, `failed`, `not-decoded`) and `fullTextError`.

### How to use

1. Enter **search queries** (Google News operators work: `"exact phrase"`, `OR`, `-word`, `site:bbc.com`, `intitle:`) and/or pick **topics** (World, Nation, Business, Technology, Entertainment, Sports, Science, Health).
2. Choose the **language and country** (default `en-US` / `US`) and add more editions if needed, e.g. `ar:SA`, `de:DE`, `en-GB:GB`.
3. Pick a **time range** (past hour, 24 hours, 7 days, 30 days, year) or exact **dates**.
4. Optional: turn on **full text**. Click **Start**, then export to JSON, CSV, Excel or send to Google Sheets, Make, Zapier or n8n.

#### Input example

```json
{
  "queries": ["openai", "\"electric vehicles\" site:reuters.com"],
  "topics": ["TECHNOLOGY"],
  "language": "en-US",
  "country": "US",
  "additionalLocales": ["ar:SA"],
  "timeRange": "7d",
  "maxArticlesPerQuery": 200,
  "decodeUrls": true,
  "extractFullText": true
}
```

#### Output example

```json
{
  "title": "NASA Seeks US Industry Plans for Commercial Space Stations",
  "publisher": "NASA (.gov)",
  "publishedAt": "2026-10-09T18:13:04Z",
  "url": "https://www.nasa.gov/news-release/nasa-seeks-us-industry-plans-for-commercial-space-stations/",
  "decodeFailed": false,
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMimAFBVV95cUxPemZaQm9…",
  "publisherUrl": "https://www.nasa.gov",
  "description": "NASA is advancing American leadership in space by taking the next step toward a future where commercial space stations lead the way in low Earth orbit. On",
  "query": "nasa",
  "topic": null,
  "locale": "en-US:US",
  "position": 1,
  "dateWindow": null,
  "relatedArticles": [],
  "articleTitle": "NASA Seeks US Industry Plans for Commercial Space Stations",
  "author": "Jessica Taveau",
  "articlePublishedAt": null,
  "articleText": "NASA is advancing American leadership in space by taking the next step toward a future where commercial space stations lead the way in low Earth orbit. On Friday, the agency released its final Request for Proposals inviting industry to submit plans for the …",
  "topImage": "https://www.nasa.gov/wp-content/uploads/2022/02/nasa_meatball_large.jpg",
  "language": "en-US",
  "paywalled": false,
  "textLength": 2897,
  "fullTextStatus": "ok",
  "fullTextError": null,
  "scrapedAt": "2026-10-10T14:41:25+00:00"
}
```

### Pricing

Pay-per-result, platform usage included:

| Event | Price |
|---|---|
| Article saved | **$0.001** ($1 per 1,000) |
| Full text extracted (only when *Extract full article text* is on and text was found) | **+$0.002** |

1,000 articles with URLs cost **$1**; with full text at most **$3**. Articles from paywalled or bot-protected sites, where no text could be read, are charged only the base $0.001. Cap any run with *Max total charge* — the scraper stops cleanly when it's reached.

### FAQ

**Why does Google News give only ~100 results?** Each Google News RSS feed is capped at about 100 articles. With a time range of 1 day or longer, or with dates set, the scraper detects a full feed and repeats the search for each single day (newest first), so busy topics return hundreds or thousands of articles. Day boundaries follow Google's own (US Pacific time). Topic headline feeds can't be split — they return Google's current selection.

**How are the URLs decoded?** Old-style Google News links contain the URL and are decoded instantly. Current links are encrypted, so the scraper asks Google News itself to resolve them (the same request the Google News web app makes), in batches. In our tests 420 of 420 links (100%) resolved, including Arabic editions; any that don't are flagged with `decodeFailed: true`.

**Why is full text missing for some articles?** Some publishers (e.g. many large paywalled newspapers) block automated visitors or show only a preview. Those articles still come with title, source, date and URL, have `fullTextStatus` set to `blocked` or `paywalled`, and are not charged the full-text fee. A residential proxy can help for blocked sites.

**Can I monitor news on a schedule?** Yes — schedule the actor with time range *Past 24 hours* (or *Past hour*) and send results to a webhook, Google Sheets or Slack.

**Is it legal?** The scraper reads Google News' public RSS feeds and publicly accessible article pages, like a news reader. Article texts are copyrighted by their publishers: use them for analysis, monitoring and research, and check the publishers' terms before republishing content.

### Related tools by DataKiln

- [Telegram Channel Scraper — posts, views & reactions from public channels](https://apify.com/datakiln/telegram-channel-scraper)
- [YouTube Channel Email Finder — creator emails, socials & stats](https://apify.com/datakiln/youtube-channel-email-finder)
- [YouTube Comments Scraper — comments, replies & likes](https://apify.com/datakiln/youtube-comments-scraper)
- [Podcast Email Finder — host emails from Apple Podcasts RSS feeds](https://apify.com/datakiln/podcast-email-finder)
- [TikTok Profile & Email Finder — bio, stats & contacts](https://apify.com/datakiln/tiktok-profile-email-finder)

### Support

Missing a field, an edition that doesn't work or a site whose text isn't extracted well? Open an issue on the **Issues** tab — we usually reply within 24 hours.

# Actor input Schema

## `queries` (type: `array`):

Keywords to search on Google News, one per line. Google News operators work: "exact phrase", OR, -exclude, site:reuters.com, intitle:word, when:2d, after:2026-01-01, before:2026-02-01 (if you add your own date operators, the date settings below are not applied to that query).

## `topics` (type: `array`):

Google News topic sections to scrape the current headlines from. Topic feeds return Google's current selection (usually 30–70 stories) and can't be split by date.

## `language` (type: `string`):

Google News edition language (hl), e.g. en-US, en-GB, de, fr, es, ar, pt-BR, ja, zh-TW.

## `country` (type: `string`):

Two-letter country code of the edition (gl), e.g. US, GB, DE, FR, SA, IN, BR.

## `additionalLocales` (type: `array`):

Scrape the same queries/topics in more Google News editions, one per line as language:COUNTRY, e.g. ar:SA, de:DE, en-GB:GB, es:MX. Results are de-duplicated across editions.

## `timeRange` (type: `string`):

Only articles published in this period. With 1 day or longer, when Google's feed is full (~100 items) the scraper splits the period into single days to collect more. Ignored when Date from / Date to are set.

## `dateFrom` (type: `string`):

Only articles published on or after this day (YYYY-MM-DD). Overrides Time range. The period is split into single days so you can get far more than 100 articles per query.

## `dateTo` (type: `string`):

Only articles published on or before this day (YYYY-MM-DD). Leave empty for today.

## `maxArticlesPerQuery` (type: `integer`):

Maximum unique articles for each query or topic in each edition.

## `decodeUrls` (type: `boolean`):

Google News RSS links point to news.google.com redirects. When on, each one is resolved to the publisher's real URL (url field). Turn off for a faster run that returns only Google links.

## `extractFullText` (type: `boolean`):

Open each article on the publisher's site and extract the main text, author, publish date, top image and language. Costs extra only for articles where the text was actually extracted.

## `maxConcurrency` (type: `integer`):

How many publisher pages to download at the same time when extracting full text.

## `proxyConfiguration` (type: `object`):

Not required. Use a proxy only if Google starts rate-limiting large runs or many publisher pages come back blocked.

## Actor input object example

```json
{
  "queries": [
    "artificial intelligence"
  ],
  "language": "en-US",
  "country": "US",
  "timeRange": "7d",
  "maxArticlesPerQuery": 100,
  "decodeUrls": true,
  "extractFullText": false,
  "maxConcurrency": 10
}
```

# Actor output Schema

## `articles` (type: `string`):

All articles as JSON items.

## `overview` (type: `string`):

Overview table view.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("datakiln/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["artificial intelligence"] }

# Run the Actor and wait for it to finish
run = client.actor("datakiln/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ]
}' |
apify call datakiln/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datakiln/google-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/g56Mwl79WiIYPEeFI/builds/wb0R7TSNS8u6Oswf7/openapi.json