Google News Scraper | Articles & Full Text

Search Google News by keyword, topic, country and language. Export headlines and publisher links, then optionally enrich articles with publisher text, descriptions and images.

Quick start · Workflows · Pricing · Input reference · Output reference · API and automation · FAQ · Technical reference

What you can do

• Keywords and topics — Combine search terms, predefined topic feeds and public topic URLs.
• Publisher links and optional enrichment — Resolve Google News URLs and request full text, descriptions or images.
• Useful source controls — Filter date windows, sites and excluded words; deduplicate articles.
• Explicit extraction quality — Distinguish usable RSS rows from verified stages and unavailable publisher content.

Who uses it

• PR and brand teams: monitor mentions and competitor coverage.
• Editors and researchers: assemble topic feeds and collect publisher links.
• Data teams: send available article text to external classification or summarization workflows.

Quick start

1. Open Input, enter your target and a small result limit.
2. Click Start and inspect the dataset and run log.
3. Export JSON/CSV, or save the input as a Task for recurring runs.

{
  "queries": [
    "artificial intelligence"
  ],
  "maxArticles": 5,
  "maxItems": 5
}

Form prefills, schema defaults and empty API input can differ. Use explicit targets and limits; see the technical reference for source and default behavior.

Choose a news edition

region_language offers 84 named edition suggestions in the Input form, and custom values remain accepted. An edition combines geography and language; it does not translate publisher articles. For example, US:en, DE:de and JP:ja select distinct editions. Topic and keyword modes use the selected edition. Publisher content, images and decoded URLs remain subject to the source's availability and access controls.

Common workflows

Collect text and images

{
  "queries": [
    "artificial intelligence"
  ],
  "maxArticles": 5,
  "maxItems": 5,
  "extractFullText": true,
  "includeImages": true
}

Read a technology topic feed

{
  "topics": [
    "TECHNOLOGY"
  ],
  "maxArticles": 10,
  "maxItems": 10,
  "country": "US",
  "language": "en"
}

Search a particular publisher

{
  "queries": [
    "AI"
  ],
  "siteFilter": "reuters.com",
  "maxArticles": 10,
  "maxItems": 10
}

Output at a glance

Illustrative field subset below; these are example values, not a captured customer review or a claim about source coverage. Native and compatibility output profiles can differ.

{
  "type": "article",
  "query": "artificial intelligence",
  "title": "Illustrative news headline",
  "url": "https://example.com/article",
  "source": "Example Publisher",
  "fullText": null,
  "imageUrl": null,
  "coverage": "partial",
  "warnings": [
    "Illustrative warning: publisher enrichment was unavailable."
  ]
}

RSS source limits apply. Publisher login, paywalls and unavailable pages are not bypassed. A usable RSS article is still a delivered billable article when requested text or image enrichment is unavailable; billing is not per complete article body.

Pricing

Pay per delivered event, with automatic discounts based on your Apify subscription plan. These are result fees in USD, not separate subscriptions sold by this Actor. All tiers have the same Actor features. Apify compute, storage, transfers and any proxy usage are charged additionally.

Billable event (1,000 events)	Free	Starter	Scale	Business
News article	$0.75	$0.6	$0.45	$0.3

Platinum and Diamond currently use the Business rate. The Pricing tab is the source of truth for the active schedule and your applicable plan.

• News article: One delivered article, including usable RSS articles when optional text or image enrichment is unavailable. Error rows have no result fee. Platform usage is additional.

Example on the Free plan: 10 / 1,000 / 10,000 events cost $0.0075 / $0.75 / $7.5 in result fees, plus platform usage.

There is no additional developer setup or monthly rental fee. A run with no billable results can still consume platform resources. Start small, cap results and review run costs before scheduling a larger workload. Requested counts are ceilings, not guaranteed deliveries.

Input reference

Fields below are grouped like the Input form. Optional fields need not be supplied. Compatibility input envelopes and transformations are explained in the technical reference.

Targets and collection

Field	Type	Default	Meaning
keyword	string	Not set	Single-query compatibility input.
queries	array	Not set	One or more queries processed in one run.
timeframe	string	Not set	Compatibility timeframe.
region_language	string	"US:en"	Choose a regional edition and language, or enter a custom COUNTRY:language code. Explicit country/language inputs take precedence. Selection requests that edition; it does not translate articles.
language	string	Not set	BCP-47 language code; overrides region_language language.
country	string	Not set	Two-letter country code; overrides region_language country.
since	string	Not set	ISO date/timestamp for scheduled incremental monitoring.
resolvePublisherUrls	boolean	Not set	Resolve Google News article links to direct publisher URLs using Google's public page metadata and decode endpoint.
extractFullText	boolean	false	Optional best-effort HTTP-only full-text extraction from publisher pages.
includeImages	boolean	Not set	Optional best-effort og:image extraction from publisher pages.
deduplicateArticles	boolean	true	Deduplicate across queries by resolved publisher URL.
requestDelayMillis	integer	0	Optional delay before each HTTP attempt; zero keeps the fast concurrent path.
keywords	array	Not set	Merged with keyword and queries, with duplicate queries removed.
decodeUrls	boolean	Not set	Alias for resolvePublisherUrls; the explicit primary field takes precedence.
extractImages	boolean	Not set	Alias for includeImages; the explicit primary field takes precedence.
topics	array	Not set	WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH. Topic feeds do not support keyword search operators; since still applies.
topicUrls	array	Not set	HTTPS news.google.com/topics/ URLs. Locale comes from country/language.
extractDescriptions	boolean	false	Extract publisher meta descriptions. Unavailable descriptions remain null.
renderMode	string	"off"	Fallback renders only when HTTP extraction has no body. Always renders publisher pages. Does not solve login, paywalls or access checks. Enable extractFullText to extract bodies.
crawlerbrosInput	object	Not set	Paste the complete original input object here. Uses saved crawlerbros defaults, including dateRange=any and maxConcurrency=5. Put all settings, including optional native proxy/browser controls, inside this object. Use one envelope only. Unchanged top-level schema defaults are ignored; other outer settings are rejected. Native output and source limits still apply.
dataXplorerInput	object	Not set	Paste the complete original input object here. Uses saved defaults: timeframe=1h, extractImages=true, maxArticles=100, decodeUrls=false, Apify Proxy enabled. Explicit nested values override defaults. Proxy service charges may apply; use nested proxyConfiguration.useApifyProxy=false to disable. Use one envelope only. Unchanged top-level schema defaults are ignored; other outer settings are rejected. Native output and source limits still apply.

Filters and ordering

Field	Type	Default	Meaning
dateRange	string	Not set	Relative recency filter; ignored when dateFrom/dateTo are supplied.
dateFrom	string	Not set	YYYY-MM-DD; overrides dateRange when set.
dateTo	string	Not set	YYYY-MM-DD; use with dateFrom.
siteFilter	string	Not set	Restrict results to a publisher domain such as reuters.com.
excludeWords	array	Not set	Terms excluded from the Google News search query.

Result limits and cost

Field	Type	Default	Meaning
maxArticles	integer	20	Compatibility input. 0 uses the 100-item RSS source ceiling per query.
maxResultsPerQuery	integer	Not set	Alternative compatibility input; takes precedence over maxArticles.
maxItems	integer	1000	Maximum saved article rows across all queries in the run.
maxRequestRetries	integer	2	Retries for transient HTTP 429, 5xx, timeout, and network failures.
maxBrowserPages	integer	5	Per-run browser page cap. Rendering is sequential to bound memory; capped or blocked pages remain partial.

Connection and advanced settings

Field	Type	Default	Meaning
maxConcurrency	integer	12	Maximum concurrent source/decode/enrichment requests.
proxyConfiguration	object	Not set	Optional Apify Proxy or custom HTTP(S) proxy URLs. Routes RSS, Google publisher-link decoding, publisher HTTP and browser requests through one per-run proxy session. Initialization failure stops the run; no silent direct fallback. Proxy usage may incur provider charges.

Output reference

The table describes native fields; fields can be absent or null when the source does not provide them. Error rows and selected compatibility profiles have different shapes. Nested objects remain structured in JSON; CSV exports may serialize them.

Field	Type	Meaning
type	string	Type
query	string	Query
title	string	Title
url	string	Publisher URL
source	string	Source
domain	string	Domain
publishedAt	string	Published at
snippet	string	Snippet
fullText	['string', 'null']	Full text
imageUrl	['string', 'null']	Image URL
author	['string', 'null']	Author
language	string	Language
country	string	Country
googleNewsUrl	string	Google News URL
sourceUrl	string	Source home URL
coverage	string	Coverage
warnings	array	Warnings
retrievalMethod	string	Retrieval method
scrapedAt	string	Scraped at
errorCode	string	Error code
message	string	Message
bodyStatus	string	Extraction result: extracted_unverified is not a claim of complete source text.

API and automation

The API tab provides snippets for this Actor. For scheduled or larger collections, start a run asynchronously and poll its status instead of holding a synchronous request open. Keep your API token in an environment variable.

import os, time, requests

base = "https://api.apify.com/v2"
headers = {"Authorization": "Bearer " + os.environ["APIFY_TOKEN"]}
actor_input = {'queries': ['artificial intelligence'], 'maxArticles': 5, 'maxItems': 5}
response = requests.post(
    base + "/acts/fmbF7jYSahlHOp5Ye/runs",
    headers=headers, json=actor_input, timeout=30,
)
response.raise_for_status()
run = response.json()["data"]
for _ in range(720):
    if run["status"] not in ("READY", "RUNNING", "TIMING-OUT", "ABORTING"):
        break
    time.sleep(5)
    response = requests.get(base + "/actor-runs/" + run["id"], headers=headers, timeout=30)
    response.raise_for_status()
    run = response.json()["data"]
else:
    raise TimeoutError("Polling ended; inspect the existing run before starting another.")
if run["status"] != "SUCCEEDED":
    raise RuntimeError(run.get("statusMessage", run["status"]))
response = requests.get(
    base + "/datasets/" + run["defaultDatasetId"] + "/items",
    headers=headers, params={"format": "json", "limit": 1000, "offset": 0}, timeout=30,
)
response.raise_for_status()
rows = response.json()
print(rows)  # Paginate with offset for larger datasets; inspect per-row coverage.

Scheduled monitoring and integrations

1. Save a working input as an Apify Task.
2. Attach a Schedule in your timezone; keep runs sharing an incremental state key sequential.
3. Add a successful-run webhook or connect your downstream workflow to the dataset export/API.
4. Inspect missing/partial data before sending alerts or treating absence as removal.

Use the platform Integrations/API options to connect external workflows such as Make, n8n or a data warehouse. An external sentiment or summarization step is separate from this Actor and may have its own charges. The platform also supports Actor tools through Apify MCP; availability and client setup depend on your integration.

FAQ

Are full article bodies guaranteed?

No. Extraction depends on each publisher. Check bodyStatus, coverage and warnings before treating a body as complete.

Can I collect a topic without a keyword?

Yes. Use topics or topicUrls and set a small result cap. Source-specific limits still apply.

Will a headline-only result be charged?

Yes, if it is a usable delivered article. Optional full-text or image failure does not make the RSS article free.

Can I export without writing code?

Yes. Run from the Input form, open the dataset and use the export controls for JSON or CSV. Select your preferred output profile before running when migrating an integration.

Does a successful run guarantee every field?

No. Inspect coverage, warnings and source-specific status fields. Optional enrichment may be missing even when usable rows were delivered.

Troubleshooting

Symptom	What to check
Empty dataset	Validate the target, source availability and filters; try the small quick-start input.
Fewer rows than requested	Check source limits, pagination and filtering. Increasing a cap cannot create unavailable source records.
Partial fields	Inspect coverage/warnings and the relevant source or enrichment status.
Access or throttling errors	Check the run log and documented proxy configuration; retry at a lower request volume.
Migration mismatch	Check the explicit input/output profile and technical field mapping below.
Unexpected bill	Separate result events from platform resources; inspect enrichment, result caps and the Pricing tab.

Technical reference

Migration-friendly inputs

The Actor accepts both common input styles already used by established Google News Actors:

• keyword, maxArticles, timeframe, region_language, proxyConfiguration
• queries, maxResultsPerQuery, language, country, dateRange, dateFrom, dateTo, siteFilter, excludeWords, extractFullText, includeImages, maxConcurrency

Extensions include since for scheduled incremental monitoring, maxItems as a global output cap, explicit deduplication, source coverage, warnings, and request/retry telemetry.

Core output

Each successful article preserves the familiar fields query, title, url, source, domain, publishedAt, snippet, fullText, imageUrl, author, language, and scrapedAt. The Actor adds googleNewsUrl, country, coverage, warnings, and retrievalMethod.

In native mode, publisher URL resolution is enabled and body/image extraction is off. Optional HTTP extraction and browser rendering are described below. Explicit compatibility profiles have their own defaults.

Source model

Google News RSS currently returns up to 100 results for a search request. Publisher URL resolution uses metadata Google serves on each public Google News article page plus the public Google News decode request used by the site. No residential proxy is required by design.

Input aliases and source failures

keywords merges with keyword and queries; timeframe accepts 7d, 30d and all. decodeUrls and extractImages alias resolvePublisherUrls and includeImages; explicit primary fields win. Topic and description controls are described below. Native output is not a drop-in data_xplorer output contract: image/imageUrl, metadata and publishedTimestamp differ.

All-query source failures now finish with platform FAILED after saving diagnostics. HTTP 429 uses bounded backoff and honors Retry-After; cooldowns longer than 30 seconds stop this attempt instead of retrying prematurely. This does not guarantee recovery from source rate limits.

Topics and descriptions

topics accepts WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH. topicUrls accepts public HTTPS Google News /topics/ URLs. Topic feeds use the selected locale; keyword search operators/dateRange do not apply to topic feeds, while since remains an article date filter. Keywords and topics may be combined. extractDescriptions=true requests publisher metadata summaries; missing metadata remains null. Access-check/error pages are not delivered as article bodies. JavaScript-only bodies can require the optional browser mode; restricted or unavailable content remains a limitation.

Browser-rendered publisher bodies

Set extractFullText=true with renderMode=fallback to render only when ordinary HTTP extraction has no body, or always to render each publisher. Default off preserves the current path. maxBrowserPages bounds browser work (default 5); one Chromium instance at a time bounds memory. Use at least 1024 MB for browser-enabled runs. Login, paywalls and access checks remain unsupported; no challenge solver is used. Missing/capped bodies stay partial.

Body quality diagnostics

bodyStatus=extracted_unverified means an article extraction candidate was returned; it does not guarantee every source paragraph was captured. coverage=complete indicates the requested extraction stages completed, not independently verified full-text completeness. Known video landing pages and short access-verification responses are not returned as article bodies. They keep feed metadata and available image/author metadata, with null fullText, partial coverage and an explicit bodyStatus/warning. Publisher fetch failures remain separate. No access challenge or paywall is bypassed. This is a conservative guard, not a universal classifier.

The body-quality guard also recognizes short verification-success waiting screens and the tested consent-dialog signature; neither is returned as article text. This remains a targeted guard and cannot certify the completeness of arbitrary publisher pages.

Article tables and subscription restrictions

Full-text extraction now retains tables selected as article content, including financial release summary and statement rows, in the extractor's pipe-separated plain-text representation. This applies to HTTP and browser-rendered extraction. It does not promise spreadsheet-grade reconstruction of merged cells or all publisher layouts. Comments remain excluded.

An article's explicit JSON-LD isAccessibleForFree=false (including a restricted hasPart) now returns fullText=null, coverage=partial, and bodyStatus=publisher_paywall_or_subscription_required. Available metadata is retained. This check runs before extraction and browser fallback, and again on rendered HTML. It does not fetch subscription content or use an account. Generic newsletter promotions and unrelated recommended articles do not trigger this specific guard. Missing or malformed access metadata is not proof of free access; other paywalls may remain undetected.

Optional proxy routing

Use proxyConfiguration to select a route. Enable Apify Proxy with {"useApifyProxy": true} (optionally set apifyProxyGroups and apifyProxyCountry), or provide {"proxyUrls": ["http://HOST:PORT"]}. Authenticated custom HTTP/HTTPS proxy endpoints are supported. SOCKS endpoints are rejected. Omitted or disabled configuration preserves the existing path.

The Apify SDK resolves one proxy URL with a stable googlenews session per run. RSS requests, Google link metadata/decode POSTs, publisher HTTP requests, and browser rendering all use that URL. HTTP proxy routing ignores environment proxy bypass settings. Browser username/password are passed separately from its proxy server address. Initialization failure stops before collection with PROXY_CONFIGURATION_FAILED; later proxy failures do not retry directly. OUTPUT.requestRouting and SOURCE_DIAGNOSTICS.requestRouting report proxy or direct, without proxy URLs or credentials. Configured proxy transport errors are reported using exception types, without raw transport exception messages. Proxy provider usage may incur separate charges; it does not guarantee access to publishers or change login/paywall/challenge restrictions.

Paste an existing competitor input

Wrap the original JSON object once in crawlerbrosInput or dataXplorerInput. Use exactly one envelope. For example:

{"crawlerbrosInput": {"queries": ["artificial intelligence"]}}

{"dataXplorerInput": {"keywords": ["artificial intelligence"]}}

The second example preserves the saved Data Xplorer proxy default (enabled). To run directly, include "proxyConfiguration": {"useApifyProxy": false} inside that object. All proxy group/country/custom-URL settings are resolved from the nested object, not from ignored top-level defaults.

Explicit entry	Saved declared defaults used
crawlerbrosInput	dateRange=any, maxResultsPerQuery=20, language=en, country=US, extractFullText=false, includeImages=false, maxConcurrency=5
dataXplorerInput	timeframe=1h, extractImages=true, maxArticles=100, region_language=US:en, decodeUrls=false, extractDescriptions=false, empty keyword/topic/URL lists, proxyConfiguration={"useApifyProxy":true}
No envelope	Existing native defaults, including dateRange=1d, images off, publisher resolution on, concurrency 12

Defaults are based on the declared public input schemas for crawlerbros and data_xplorer. UI prefill values are not API defaults. Supply at least one query or topic; an empty envelope is rejected instead of inventing a search. Crawlerbros requires its queries array. This does not establish unspecified competitor runtime defaults or reproduce their output schema.

Explicit nested values take precedence; supported native aliases and runtime controls can also go inside the envelope. If both an explicit primary field and its alias are supplied, existing native primary-field precedence applies. Do not put business settings outside the envelope: unchanged top-level Schema defaults are ignored (including manually supplied values equal to those defaults), and other outer settings are rejected. This isolates each profile from platform injection of native defaults. Native output fields, the 100-result per-query ceiling, HTTP/render/access restrictions and non-competitor runtime controls stay in effect. An enabled image stage does not guarantee a publisher image exists.

OUTPUT and SOURCE_DIAGNOSTICS report inputProfile plus a safe resolvedInput summary of time window, image/decode flags, locale and limits. Proxy URLs, credentials and raw envelope contents are excluded. Invalid envelope input is rejected before collection without echoing submitted values.

Support

Open an issue from this Actor’s Issues tab with the run ID, expected result and a redacted input. Do not include tokens, cookies or private credentials. Include the output profile and affected field when reporting a migration mismatch.

Presentation update

The current listing adds workflow examples, grouped input documentation, output-field explanations and plan-based result discounts. Source-specific limitations above still apply. This is an independent tool and is not affiliated with the source platform.