# Google News Scraper: Articles, Publishers & Real Links (`arman-bd/google-news-scraper`) Actor

Search Google News by keyword, topic or publisher across 91 country/language editions. Headlines, publishers and publish dates, with optional resolution of every article to the publisher's own URL instead of a redirect token.

- **URL**: https://apify.com/arman-bd/google-news-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** News, AI, MCP servers
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.40 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google News Scraper: Articles, Publishers & Real Links

![Google News Scraper: Headlines by keyword, topic or publisher, with the publisher's own link, not a redirect token](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/google-news-scraper.jpg)

**Google News Scraper** searches Google News by keyword, topic or publisher and returns one clean row per article: headline, publisher, publisher domain, publish date in ISO 8601, and the rank it held for your query. Every Google search operator works inside a query, and 91 country/language editions are selectable as a single value.

Turn on link resolution and `url` becomes the publisher's own article address instead of a 250-character redirect token — and `urlResolved` tells you, per row, whether that succeeded. **No API key, no quota, no proxy setup and no browser.**

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/google-news-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/google-news-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `title` | The headline. The `- Publisher` suffix Google appends is removed, since `publisher` already carries it |
| `url` | The publisher's own article URL when resolution is on, otherwise the Google link |
| `googleUrl` | Always the Google link, so you can re-resolve later without re-running the search |
| `urlResolved` | Whether `url` is the publisher's. Never leaves you guessing which one you got |
| `publisher` | Publisher name as Google files it |
| `publisherUrl` | The publisher's home page |
| `publisherDomain` | Host of `publisherUrl`, lowercased and without `www.` — the field you group by |
| `publishedAt` | ISO 8601, UTC |
| `publishedRaw` | The feed's own RFC-822 date string, unparsed, for auditing |
| `snippet` | Article summary. **`null` unless `includeArticleMetadata` is on** — see Limits |
| `thumbnailUrl` | Lead image. **`null` unless `includeArticleMetadata` is on** — see Limits |
| `query` | Which input produced the row: the query text, the topic, or the domain |
| `queryType` | `search`, `topic` or `publisher` |
| `edition` | The edition the row came from, echoed as `US:en` |
| `position` | Rank within its query |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds the edition used, every query run and its kind, the resolved date range, the filters applied, requests made, articles saved, duplicates collapsed, how many links resolved and how many did not **with the reason for each**, how many article pages returned a summary, returned nothing, or could not be read at all, inputs that were rejected and why, and any field that failed its shape check. Failures are named there rather than left to look like a quiet news day.

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `queries` | array | required | Search terms. Operators work: `"quoted phrase"`, `site:reuters.com`, `intitle:merger`, `-exclude`, `OR` |
| `topics` | array | `[]` | `WORLD`, `BUSINESS`, `TECHNOLOGY`, `ENTERTAINMENT`, `SPORTS`, `SCIENCE`, `HEALTH`. Fetched on their own; a section feed takes no search terms, so it is one feed of the last day or two — see Limits |
| `publishers` | array | `[]` | Domains such as `reuters.com`. Applied on top of each query, so they narrow a search rather than replace it. Given alone, each becomes its own search |
| `edition` | string | `US:en` | Country and language together, from one value. 91 available |
| `maxItemsPerQuery` | integer | `100` | Cap per query, and the main cost control since charging is per article |
| `dateFrom` | string | | `2026-01-15`, or relative: `7 days`, `36 hours`, `3 months` |
| `dateTo` | string | | Same formats. Ignored unless `dateFrom` is also set |
| `resolvePublisherUrl` | boolean | `false` | Return the publisher's own link. Two extra requests per article, charged separately |
| `includeArticleMetadata` | boolean | `false` | Fetch each article's own summary and image. Switches on `resolvePublisherUrl`, adds a third request per article, charged separately |
| `deduplicateBy` | string | `url` | `url`, `title` or `none` |

At least one of `queries`, `topics` or `publishers` must be non-empty; a run with all three empty fails with a named error rather than returning nothing.

```json
{
 "queries": ["apify OR \"web scraping\""],
 "edition": "GB:en",
 "maxItemsPerQuery": 50,
 "dateFrom": "14 days",
 "resolvePublisherUrl": true
}
```

### Output

One record per article. Every field is present on every record, and `null` always means the value is not available for that article — never that the column was quietly dropped. Where a `null` came from an attempt that failed rather than from an article that simply has no such value, `RUN_SUMMARY` says so and counts it separately.

```json
{
 "title": "OpenAI says it slowed Astra model development over security concerns",
 "url": "https://techcrunch.com/2026/08/07/openai-says-it-slowed-astra-model-development-over-security-concerns/",
 "googleUrl": "https://news.google.com/rss/articles/CBMipgFBVV95cUxNd3B2a2NoaGtOdmJ5WEhERFMwbTZOX3p1MHplbG1ocnRsa25kaDFIRXU5SjVwVmFzRW1ZTWJWZ1dnYmljZ0pMVUZ1SE5RZXpaVy0ydnpjd2lVaHpuTUZEckZDSUI0MGF5N1dYZkItcmNiQkdiVlBPZFZwRENjNzIwYkVQdml5b0Z0c1Rzd3BTMXFWQ2t5N01qQ2w0T1N2bmN4US1Ra2pR?oc=5",
 "urlResolved": true,
 "publisher": "TechCrunch",
 "publisherUrl": "https://techcrunch.com",
 "publisherDomain": "techcrunch.com",
 "publishedAt": "2026-08-07T16:31:00.000Z",
 "publishedRaw": "Fri, 07 Aug 2026 16:31:00 GMT",
 "snippet": null,
 "thumbnailUrl": null,
 "query": "openai",
 "queryType": "search",
 "edition": "US:en",
 "position": 1,
 "scrapedAt": "2026-08-09T01:12:44.000Z"
}
```

### Use cases

- **Brand and PR monitoring.** Track every mention of a company across 91 national editions, on a schedule, and diff successive runs on `googleUrl`.
- **Competitor press alerts.** One query per rival with `dateFrom: "1 days"` on an hourly schedule, and you see coverage the hour it lands.
- **Dated corpus building.** For a search query, `maxItemsPerQuery` above 100 slides the date range into windows automatically, so a year of one subject comes out as one run rather than fifty.
- **Retrieval for agents.** Current headlines with a real publisher link is exactly the shape a retrieval step wants, and the whole thing is one API call.
- **Media landscape analysis.** Group by `publisherDomain` to see which outlets own a story, and compare the same query across editions to see how a story travels.

### Limits and behaviour

- **`snippet` and `thumbnailUrl` are `null` unless `includeArticleMetadata` is on.** Google News publishes no summary and no image of any kind alongside a headline: not a thumbnail, not a caption, not a first paragraph. The one thing that looks like a summary is a restatement of the headline and the publisher name, which is `title` and `publisher` over again. Mapping it through would give you a populated-looking column with nothing in it, so this Actor does not. With `includeArticleMetadata` on, both fields are read from the article's own page instead.
- **A publisher's own summary is often just the headline again.** Many outlets set their page description to the title verbatim. The field is best-effort by nature, and stays `null` when the page declares nothing. If a page could not be read at all — a block, a timeout, a redirect to a consent wall — that is counted apart from "declared nothing" in `RUN_SUMMARY`, with the reason, so an outage never reads as publisher habit. Neither case is charged.
- **Topic feeds are standing sections, so they behave differently from a query.** A section takes no search terms, which means the date range cannot travel with the request: it is applied to the articles after the feed is read, and `RUN_SUMMARY.dateRange.topicArticlesOutOfRange` reports how many that removed. A section also cannot be split into date windows, so a topic returns one feed no matter how high `maxItemsPerQuery` goes, and it reaches back only as far as the section itself carries — usually a day or two.
- **Roughly 100 articles per query per request.** Above that the run splits your date range into windows and requests each one, newest first; `RUN_SUMMARY.perQuery[...].windows` reports how many it used. A larger cap therefore costs proportionally more time, and a narrow date range cannot be split beyond one window per day.
- **Resolution is best-effort and says so.** Every article that resolves gets the publisher's URL and `urlResolved: true`. Every article that does not still ships, keeps the Google link in `url`, and is **not charged** for the failed attempt.
- **Google News indexes headlines, not article bodies.** There is no full text here, and no way to search inside one.
- **Syndicated duplicates are a property of the source.** A wire story runs at a dozen outlets under near-identical headlines. `deduplicateBy: "url"` keeps them all; `"title"` collapses them, at the cost of genuinely distinct regional rewrites.
- **An edition changes which articles rank, not only their language.** The same query in `GB:en` and `AU:en` overlaps around 60%, and `CA:fr` against `FR:fr` around 20%. Country and language are set from one value so they cannot be made to contradict each other.
- **Topic feeds take no search terms.** They are standing sections, so `topics` is fetched separately and is not combined with `queries` or `publishers`.
- **A bad input never ends the run.** An unrecognised topic or a publisher that is not a domain is recorded in `RUN_SUMMARY.rejectedInputs` and the rest proceeds. The run fails only if everything failed.
- **Public data only.** No credentials, and nothing that works around access controls.

### FAQ

**Why do the links look like `news.google.com/rss/articles/…`?** That is how Google publishes them, and it is what every article link is until it is resolved. Set `resolvePublisherUrl: true` and `url` becomes the publisher's own address; `googleUrl` always keeps the original either way.

**Why is `snippet` empty?** Because the news feed has no summary field. Turn on `includeArticleMetadata` and it is read from the article's own page. See Limits.

**Do search operators work?** Yes — quoted phrases, `site:`, `intitle:`, `-exclusion` and `OR`, exactly as you would type them into Google News. `publishers` is a convenience on top of `site:`, so the two compose.

**Why does changing the edition change my results?** Each edition is a different national front page with its own publisher mix and ranking, not a translation layer. Pick the edition your audience reads.

**Can I get more than 100 articles for one query?** Yes. Raise `maxItemsPerQuery` and the run slides the date range into windows automatically. Give it a `dateFrom` so it knows how far back to reach.

**Do I need a Google API key or a news licence?** No. You supply no credentials.

**How do I run this on a schedule?** Apify schedules, or your own cron against the API. Keep the dataset and diff runs on `googleUrl`, which is stable per article.

**Can I plug it into something else?** Yes. Apify API, the client libraries, webhooks, scheduled runs, dataset exports to JSON, CSV or Excel, or MCP. The output is structured JSON.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~google-news-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "queries": ["\"supply chain\" intitle:disruption"],
 "edition": "GB:en",
 "maxItemsPerQuery": 50,
 "dateFrom": "7 days",
 "resolvePublisherUrl": true
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/google-news-scraper').call({
 queries: ['apify OR "web scraping"'],
 edition: 'US:en',
 maxItemsPerQuery: 100,
 dateFrom: '14 days',
 resolvePublisherUrl: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) {
 console.log(`${a.publishedAt} ${a.publisher} — ${a.title}`);
 console.log(a.urlResolved ? a.url : `unresolved: ${a.googleUrl}`);
}
```

# Actor input Schema

## `queries` (type: `array`):

What you would type into Google News. Operators work: "quoted phrases", site:reuters.com, intitle:merger, -exclude, OR. Each query is searched independently, so two queries cost roughly twice one.

## `topics` (type: `array`):

Standing Google News sections. These are fetched on their own and are not combined with your queries, because a section feed takes no search terms. A section is one feed holding roughly the last day or two, so it cannot be split into date windows and a date range is applied to its articles after the feed is read rather than narrowing what comes back.

## `publishers` (type: `array`):

Domains such as reuters.com or ft.com. Applied as a site restriction on top of each query, so it narrows a search rather than replacing it. Given on their own, with no queries, each domain becomes its own search.

## `edition` (type: `string`):

Which national edition of Google News to read. Country and language are set together from this one value: supplied separately they can contradict each other, and a contradictory pair silently returns US English. 91 editions are available.

## `maxItemsPerQuery` (type: `integer`):

Cap on articles returned for each query. One feed request tops out near 100; above that the run splits your date range into windows and requests each, which costs proportionally more time. Topic sections cannot be split that way and stay at one feed. This is the main cost control, since charging is per article.

## `dateFrom` (type: `string`):

Oldest article to return. Absolute as 2026-01-15, or relative as "7 days", "36 hours", "3 months".

## `dateTo` (type: `string`):

Newest article to return, in the same formats. Ignored unless "Published after" is also set, because the source needs a start to bound a range.

## `resolvePublisherUrl` (type: `boolean`):

Return the publisher's own article link instead of the Google redirect token. Costs two extra requests per article, is charged separately, and is off by default so a cheap run stays cheap. Articles that cannot be resolved still ship, flagged urlResolved: false, and are not charged for the attempt.

## `includeArticleMetadata` (type: `boolean`):

Read each article's own page for its summary and lead image. The news feed carries neither, so this is the only way to get them: snippet and thumbnailUrl stay null without it. Needs the publisher URL, so it switches URL resolution on, adds a third request per article, and is charged separately. Publisher summaries are often just the headline again. Pages that declare nothing and pages that could not be read are counted apart in the run summary, and neither is charged.

## `deduplicateBy` (type: `string`):

Wire stories are carried by many outlets at once. "url" removes only the identical article arriving twice. "title" also collapses syndication, which is often what you want and sometimes throws away distinct regional rewrites. "none" keeps everything.

## Actor input object example

```json
{
  "queries": [
    "\"climate policy\"",
    "apify OR \"web scraping\""
  ],
  "topics": [],
  "publishers": [
    "reuters.com",
    "apnews.com"
  ],
  "edition": "US:en",
  "maxItemsPerQuery": 100,
  "dateFrom": "14 days",
  "dateTo": "2026-02-01",
  "resolvePublisherUrl": false,
  "includeArticleMetadata": false,
  "deduplicateBy": "url"
}
```

# Actor output Schema

## `items` (type: `string`):

Every article the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "artificial intelligence"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/google-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["artificial intelligence"] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/google-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "artificial intelligence"
  ]
}' |
apify call arman-bd/google-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/google-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/IVGGtcOAkaqGRlO8O/builds/amzic5ypYtaaezZJw/openapi.json