# Google News Scraper: Article Text and New-Only Monitor (`danthedataman/google-news-article-ledger`) Actor

Scrape Google News by keyword, topic or publisher. Get headlines, dates, resolved publisher URLs, best-effort article text and new-only results for news monitoring.

- **URL**: https://apify.com/danthedataman/google-news-article-ledger.md
- **Developed by:** [Eli J](https://apify.com/danthedataman) (community)
- **Categories:** News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.54 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google News Article Ledger

Google News scraper for keyword searches, Google News topics and publisher sites. Each row is one article: headline, publisher, publication date, the resolved publisher URL and, when the publisher page can be read, readable article text, description and image. For scheduled monitoring, supply a successful full-snapshot dataset or an accumulated list of seen URLs. Each run returns only articles absent from that state; update the state between runs to avoid repeat delivery.

Typical uses: news monitoring for a company, product or topic; collecting headlines and article text for research or sentiment analysis; feeding a daily digest without duplicates.

### Try it

Click Start with the prefilled input. It searches Google News for `OpenAI` in US English and returns up to 20 articles with resolved URLs and best-effort article text. The prefill is a form example only. An API call with `{}` returns no rows and a guidance message.

### What you can search

- Search terms, for example `OpenAI` or `climate policy`, and Google News search URLs.
- Topics: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH, or a Google News topic/section URL.
- Publisher sites, for example `bbc.com`.

Mix source types in one run and choose a Google News edition with `country` and `language`; source availability and locale support can vary. Recency windows and excluded words apply to search and publisher inputs, while topics keep Google's curated selection. `siteFilter` applies only to search inputs. Exact `since`/`until` filters apply before limits and billing, but only search and publisher inputs page back through older date windows.

### Three modes

| Mode | Settings | What you get |
| --- | --- | --- |
| Article text (default) | `resolveUrls` true, `extractArticleText` true | Resolved publisher URL plus best-effort article text, publisher description, image, author, section and keywords when the publisher page can be read. |
| Resolved metadata | `extractArticleText` false | Resolved publisher URLs and feed metadata, no publisher page reads. Suited to large pulls. |
| Feed only | `resolveUrls` false, `extractArticleText` false | Google News links and feed metadata only. |

### Input examples

Several search terms:

```json
{"startInputs":["OpenAI","climate policy"],"maxItems":40,"maxArticlesPerInput":20}
```

Topics and a publisher in one run:

```json
{"startInputs":["TECHNOLOGY","bbc.com"],"maxItems":30}
```

German news for September 2026, metadata only, no per-input cap:

```json
{"startInputs":["artificial intelligence"],"country":"DE","language":"de","since":"2026-09-01","until":"2026-09-30","maxItems":1000,"maxArticlesPerInput":0,"extractArticleText":false}
```

Filtered search:

```json
{"startInputs":["energy"],"siteFilter":["bbc.com","theguardian.com"],"excludeWords":["opinion","sponsored"],"timeframe":"7d","maxItems":20}
```

New articles only, against a saved full snapshot:

```json
{"startInputs":["OpenAI"],"baselineDatasetId":"YOUR_FULL_SNAPSHOT_DATASET_ID","maxItems":20}
```

### Input fields

| Field | Type | What it does |
| --- | --- | --- |
| `startInputs` | array | Search terms, topics, Google News search or topic URLs and publisher domains. Empty means no requests. |
| `country` | string | Two-letter Google News country code. Default US. |
| `language` | string | Language code, including variants such as zh-Hant. Default en. |
| `timeframe` | string | all, 1h, 6h, 1d, 7d, 30d, 1m or 1y; default all. Applies to search and publisher inputs. Explicit dates override it. Topics keep Google's curated window. |
| `since` | string | Inclusive start date or zoned datetime. Search and publisher inputs walk back through date windows to reach it. |
| `until` | string | Inclusive end date or zoned datetime; a calendar date includes the whole UTC day. |
| `maxItems` | integer | Maximum articles delivered in the run. Default 100, accepted up to 50,000; a cap, not a promised yield. |
| `maxArticlesPerInput` | integer | Maximum distinct articles selected per input before new-only suppression, known URLs included. Default 100; 0 removes this cap within the run's time budget. |
| `resolveUrls` | boolean | Default true. False keeps Google News links and reads no publisher pages. |
| `extractArticleText` | boolean | Default true. Reads publisher pages for text and metadata. Needs `resolveUrls` true. |
| `siteFilter` | array | Publisher domains for search inputs. Publisher-domain inputs ignore it; it does not filter topics. |
| `excludeWords` | array | Words to exclude from search and publisher inputs. Topics keep Google's selection. |
| `alreadySeenUrls` | array | Canonical URLs you already have, from the same `resolveUrls` setting. Matching articles are skipped and not charged. |
| `baselineDatasetId` | string | Read-only dataset from a successful full-snapshot run. Enables new-only output. Never use a new-only run's dataset here. |

### Output

One row per distinct article URL, 31 fields. Abbreviated example from article-text mode (illustrative values):

```json
{
  "title": "Example article headline",
  "source": "Example Publisher",
  "publisherDomain": "example.com",
  "url": "https://example.com/news/article",
  "googleNewsUrl": "https://news.google.com/rss/articles/EXAMPLE",
  "publishedAt": "2026-10-06T09:00:00Z",
  "publisherPublishedAt": "2026-10-06T08:59:00Z",
  "description": "Publisher description, or the feed summary.",
  "body": "Readable publisher article text when available.",
  "wordCount": 6,
  "imageUrl": "https://example.com/images/lead.jpg",
  "author": "Writer A, Writer B",
  "section": null,
  "keywords": [],
  "bodyExtracted": true,
  "bodyFailureReason": null,
  "sourceInput": "OpenAI",
  "country": "US",
  "language": "en",
  "scrapedAt": "2026-10-06T19:00:00Z"
}
```

| Field | Meaning |
| --- | --- |
| rowType, sourceKind, sourceInput, sourceRssUrl | Always `news-article`; query, topic or publisher; the input you supplied; the feed URL that found the row. |
| title, source, publisherDomain | Headline, publisher name and domain from the feed. |
| url, publisherUrl | With `resolveUrls` true, both hold the resolved publisher URL. In feed-only mode, `url` holds the Google News link and `publisherUrl` is null. |
| canonicalUrl | The resolved publisher URL, or the Google News link in feed-only mode, with fragments, trailing slashes and common tracking parameters removed. It is the identity used for new-only output, not a verified publisher rel=canonical. |
| googleNewsUrl, guid, position | Google's link, feed ID and position in its feed; position can restart on another date window. |
| publishedAt, publishedAtRaw, publishedTimestamp | Google's publication time: UTC, the raw feed string, and milliseconds. |
| publisherPublishedAt, dateDisagreement | The publisher's own time when available, and whether it differs from Google's. |
| description | With article text on, a confirmed publisher description when the page has one; otherwise the feed summary as plain text, or null. |
| body, bodyHtml | Publisher article text and rebuilt paragraph HTML after quality checks, or null. Bodies under 400 characters, paywalled pages and unconfirmed selections are refused. Not a guarantee of the complete article. |
| wordCount | Words in `body`, or null. |
| imageUrl, author, section, keywords | From the publisher page when read and available; otherwise null or empty. Author names supplied as a list of strings are joined with ", "; other non-string names return null. This does not guarantee that every author is returned. |
| bodyExtracted, bodyFailureReason | Whether body extraction succeeded, and any body or metadata-identity failure reason. `article_identity_unconfirmed` can describe refused metadata even when `bodyExtracted` is true. `not_requested` means extraction was off; null means nothing failed. |
| changeStatus | `new` in new-only runs. |

Export as JSON, CSV or Excel.

### New articles only (monitoring)

1. Run once without `baselineDatasetId` or `alreadySeenUrls`. Keep the dataset from that successful full-snapshot run.
2. Pass that dataset as `baselineDatasetId`, or supply your accumulated `alreadySeenUrls`. When both are supplied, both filters apply.
3. For later runs, add every delivered `canonicalUrl` to your seen-URL list, or take another successful full snapshot. Never use a new-only run's dataset as the next baseline.
4. Keep the same `resolveUrls` setting. Google News links and publisher URLs are different identities; changing modes does not translate existing state.

Use an Apify task and schedule for recurring monitoring. Each run also writes a SNAPSHOT key-value record with the current selected window, known URLs included; known rows carry no new body. Limits, failures or the time budget can shorten that window. It is not a complete archive or a baseline dataset, and removed or changed articles are not reported.

### Pricing

Pay per delivered article, plus $0.00005 per run start ($0.05 per 1,000 starts). Free and Bronze plans pay $1.50 per 1,000 articles, Silver $1.20, and Gold, Platinum and Diamond $0.54. Already-seen articles, errors and empty results have no article charge; a started run still pays the start event. A maximum-charge limit stops the run before the next batch; a batch already in flight finishes first. The Store pricing panel shows the current price.

### Limits and known gaps

- Publisher text, images and descriptions are best-effort. Five cohorts of eight prefill runs on 2026-10-10, on builds with the same runtime code, each produced 160 rows: 0.1.25 at 21:58-22:07 UTC and 0.1.26 at 22:33-22:35, both at 512 MB, and 0.1.27 at 22:41-22:45, 0.1.28 at 22:54-23:08 and 0.1.29 at 23:15-23:29, all three at 1024 MB. Images appeared in 48.1% to 71.9% of rows. Descriptions appeared in 51.9% to 76.9% of rows. A description counts when it differs from the feed's headline-and-source text. Coverage depends on the share of results from publishers that refuse automated reads, such as WSJ, NYT, FT and Bloomberg, and that share varies with the news set. The 75% image target is not met. In the 0.1.28 cohort, image misses came from HTTP 401 or 403 responses, read timeouts and unconfirmed article identity. Browser retries supplied publisher fields or text on 1 of 38 attempts in the 0.1.25 cohort and 9 of 77 in the 0.1.28 cohort.
- Startup time varies. On 2026-10-10 at 512 MB, five 20-row feed-only runs had a 9.729-second median, slower than the 4.945-second median measured on 2026-10-07 with the previous, smaller image. Five more runs had a 4.043-second median; the ten-run median was 4.768 seconds. Two runs spent 14.578 and 16.521 seconds pulling the image, and the slowest took 36.758 seconds in total. Feed-only mode never starts the browser, but the browser-capable image is larger. These are samples, not a guaranteed startup time.
- Memory and timeouts. The default memory is 1024 MB. Use at least 512 MB and a 300-second timeout. Heavy publisher-text runs used most of a 512 MB allocation: on 2026-10-10, four of eight article-text runs on build 0.1.25 peaked at 511.5 to 512 million bytes. All eight started the browser fallback and succeeded. The Actor stops starting new HTTP work after 240 seconds with article text on, or 270 seconds otherwise. A stalled browser startup or shutdown can still delay storage and use up the platform timeout.
- Volume. Feeds are finite windows. On 2026-10-07 a 512 MB metadata run delivered 1,000 articles in 219.597 seconds; a run asking for 50,000 reached its time budget at 975 articles in 281.311 seconds. Size metadata requests around 1,000 rows. A 2026-10-06 text run with 40 concurrent reads delivered 271 articles, 112 with extracted bodies, in 255.109 seconds; this version allows 50. These dated examples are not guaranteed throughput.
- Not supported: full-coverage story clusters, entity extraction, inline thumbnails and Google search controls such as spelling correction or country restriction.

### How it works

Publisher pages are read over HTTP first. After three failed attempts of a retryable kind, or an HTTP 401, one headless Chromium read is tried with the same article identity and field checks. HTTP 404 and other final 4xx answers, an exhausted time budget, and pages refused for quality or identity never start the browser. Metadata and feed-only modes never read publisher pages. Errors go to a separate ERRORS record and never become dataset rows; COVERAGE reports browser attempts and rescues.

Google News robots.txt disallows the RSS and resolver paths (checked 2026-10-06). The Actor reads public pages without logging in and does not claim permission from the target.

### Support

Open an issue on the Actor's Issues tab with the run link.

# Changelog

This Actor's version history is a separate document: https://apify.com/danthedataman/google-news-article-ledger/changelog.md

# Actor input Schema

## `startInputs` (type: `array`):

Mixed queries, predefined topic names, Google News search/topic URLs or publisher domains. Omitted/null/\[] makes no requests.

## `country` (type: `string`):

Two-letter country code. Omitted/null means US.

## `language` (type: `string`):

Google News language code, including locale variants. Omitted/null means en.

## `timeframe` (type: `string`):

Recency operator for query/publisher feeds. Omitted/null means all. Explicit dates override it; topic feeds keep Google's window.

## `since` (type: `string`):

Inclusive ISO date or zoned datetime. A since date enables backward day slicing for query/publisher feeds. No filter when omitted/null.

## `until` (type: `string`):

Inclusive ISO date or zoned datetime. A date includes its whole UTC day. No upper publication filter when omitted/null.

## `maxItems` (type: `integer`):

Maximum delivered rows, not a guaranteed yield. Omitted/null means 100; zero makes no requests.

## `maxArticlesPerInput` (type: `integer`):

Maximum distinct current canonical URLs selected per input before seen-URL suppression. Omitted/null means 100; zero removes this cap within the run budget.

## `resolveUrls` (type: `boolean`):

Omitted/null means true. False returns Google News links and feed metadata only, with no signatures, decoding or publisher reads; requires extractArticleText=false.

## `extractArticleText` (type: `boolean`):

Omitted/null means true. True requires resolveUrls=true. Fetch publisher body and metadata; false keeps feed metadata only. Inaccessible or unconfirmed bodies remain null with a reason.

## `siteFilter` (type: `array`):

Domains OR-ed with site: operators for query inputs; ignored for publisher-domain inputs. No restriction for omitted/null/\[].

## `excludeWords` (type: `array`):

Quoted negative search operators for query/publisher inputs. Omitted/null/\[] adds no exclusions; topic feeds retain Google's selection.

## `alreadySeenUrls` (type: `array`):

Suppress previously delivered canonical URLs in the same resolution mode. Keep an accumulated list for successive monitor runs; no article-row charges for matches.

## `baselineDatasetId` (type: `string`):

Read-only dataset from a successful full snapshot run. Emits only unseen canonical URLs. Unreadable or malformed baselines fail before source requests; never supply a monitor delta.

## Actor input object example

```json
{
  "startInputs": [
    "OpenAI"
  ],
  "country": "US",
  "language": "en",
  "maxItems": 20,
  "maxArticlesPerInput": 20,
  "resolveUrls": true,
  "extractArticleText": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `snapshot` (type: `string`):

No description

## `coverage` (type: `string`):

No description

## `errors` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startInputs": [
        "OpenAI"
    ],
    "country": "US",
    "language": "en",
    "maxItems": 20,
    "maxArticlesPerInput": 20,
    "resolveUrls": true,
    "extractArticleText": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("danthedataman/google-news-article-ledger").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startInputs": ["OpenAI"],
    "country": "US",
    "language": "en",
    "maxItems": 20,
    "maxArticlesPerInput": 20,
    "resolveUrls": True,
    "extractArticleText": True,
}

# Run the Actor and wait for it to finish
run = client.actor("danthedataman/google-news-article-ledger").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startInputs": [
    "OpenAI"
  ],
  "country": "US",
  "language": "en",
  "maxItems": 20,
  "maxArticlesPerInput": 20,
  "resolveUrls": true,
  "extractArticleText": true
}' |
apify call danthedataman/google-news-article-ledger --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,danthedataman/google-news-article-ledger"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mFgXMbGP25JaMPzsV/builds/w569Q2UXVR4Yg9P5O/openapi.json