Google News Scraper: search, topics, publishers, full text, monitor

Scrape Google News into clean JSON/CSV rows: headline, the real publisher URL (Google's redirect links are decoded for free), source, ISO publication time, snippet, and optionally the full article text with author, excerpt and lead image. Search by keyword, scrape the eight Google News sections or any Google News page URL, restrict to publishers, pick a time window in any of Google's 100+ country/language editions, import hundreds of queries from a CSV or Google Sheet, and run it on a schedule in monitor mode so each run returns only articles that are new since the last one.

Pure HTTP: no browser, no API key, no login, no cookies. A run of 100 articles takes a few seconds.

What you get

One row per article. Fields from a real run ({"queries": ["artificial intelligence"]}):

Field	Example	Notes
title	Meta earnings live updates: AI spending, ad revenue and stock reaction	publisher suffix removed
url	https://www.cnbc.com/2026/10/21/meta-earnings-q3-2026-live-updates.html	real publisher URL (resolveUrls, on by default)
urlResolved	true	false when decoding is off or failed; url then holds the Google link
googleNewsUrl	https://news.google.com/rss/articles/CBMie0FV…?oc=5	Google's redirect link
guid	CBMie0FVX3lxTE5HVnM1…	Google's article id, stable across runs (use it to dedupe)
source	CNBC	publisher name
sourceUrl / sourceDomain	https://www.cnbc.com / cnbc.com
publishedAt	2026-10-21T16:00:00.000Z	ISO 8601
snippet	null	Google's snippet when it differs from the title; topic feeds carry the related-coverage headlines here
imageUrl	https://image.cnbcfm.com/api/v1/image/108307357-….jpeg	lead image (includeArticleContent)
query / topic / inputKind	artificial intelligence / null / search	which input produced the row (search, topic, publication, story, local)
feedTitle	"artificial intelligence" - Google News
language / country	en / US	edition used
isNew	true	monitor mode only
scrapedAt	2026-10-21T16:16:05.029Z

With includeArticleContent: true each row also has:

Field	Example
content	On SpaceX's earnings call late Tuesday, CEO Elon Musk, who also runs Tesla, took a break from talking about financial results to tell investors that it would be a "good idea" to merge… (full readable text)
wordCount	728
author	Lora Kolodny
excerpt	A day after Elon Musk's SpaceX reported third-quarter results, Tesla is scheduled to update investors.
articleTitle / siteName / canonicalUrl	title and site name as published, canonical link
contentExtracted	true; false with contentError (HTTP 403, no article text found (paywall, consent wall or JS-only page))

Legacy field names from version 1 (sourceName, publishedDate, publishedDateISO, description, searchQuery) are still present on every row, so existing integrations keep working.

Use cases

• Brand and PR monitoring: schedule the actor hourly with your brand, product and executive names as queries, turn on onlyNew, and push new coverage to Slack or a sheet through the webhook or an Apify integration.
• Competitor and market intelligence: track competitors and sector keywords across editions (country/language) and compare coverage between markets.
• Financial news feeds: company names or tickers with timeRange: "1d" and sortBy: "date"; add includeArticleContent for sentiment models that need the body, not the headline.
• RAG and LLM pipelines: clean article text with author, publishedAt and canonicalUrl, deduplicated by URL, in one dataset.
• Media and newsroom research: paste a "Full coverage" story URL to get every publisher covering one story, or a topic section URL for a niche beat.
• Local news: a Google News local page URL (/headlines/section/geo/Austin) gives a city's feed.
• Publisher watch: publications: ["reuters.com", "apnews.com"] with no query returns those outlets' newest articles.

Input

Search by keyword

{
  "queries": ["OpenAI", "Anthropic", "\"AI regulation\" -crypto"],
  "timeRange": "7d",
  "sortBy": "date",
  "maxItems": 300,
  "maxItemsPerQuery": 100
}

Google operators work inside a query: "exact phrase", OR, -exclude, site:bbc.com, intitle:merger. Google reads about the first 32 words.

Bulk import from a CSV or Google Sheet

{
  "listUrl": "https://docs.google.com/spreadsheets/d/1AbC…/edit#gid=0",
  "timeRange": "1d",
  "maxItems": 5000
}

The first column of each row is used; a header row (query, brand, company…) is skipped. Rows that are news.google.com links are treated as startUrls. Share the sheet as "anyone with the link".

Topics and Google News URLs

{
  "topics": ["BUSINESS", "TECHNOLOGY"],
  "startUrls": [
    "https://news.google.com/topics/CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB",
    "https://news.google.com/headlines/section/geo/Austin",
    "https://news.google.com/stories/<STORY_ID from a 'Full coverage' page>"
  ],
  "maxItemsPerQuery": 50
}

Topics: TOP, WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH. Any news.google.com search, topic, topic section, local, story ("Full coverage") or /rss/ URL works; the locale in the URL is kept. Publication pages (/publications/…) have no feed on Google's side; use publications instead.

Publishers, exclusions, editions

{
  "queries": ["electric vehicles"],
  "publications": ["reuters.com", "bloomberg.com", "ft.com"],
  "excludeWords": ["Tesla"],
  "language": "de",
  "country": "DE"
}

language and country select the Google News edition (DE:de, FR:fr, IN:en, BR:pt-BR, MX:es-419, JP:ja…). A combined token such as "language": "DE:de" or "regionLanguage": "DE:de" is accepted too.

Time windows

{ "queries": ["earthquake"], "timeRange": "1h" }
{ "queries": ["iPhone 18"], "timeRange": "custom", "dateFrom": "2026-09-09", "dateTo": "2026-09-16" }
{ "queries": ["OpenAI"], "publishedAfter": "2026-10-21T08:00:00Z" }

timeRange: any, 1h, 1d, 7d, 30d, 1y, custom. Past date ranges work (Google's after:/before: operators), and every row is checked against the window before it is charged. Topic and URL feeds return Google's current selection; the dates filter them.

More than 100 results per query. Google serves about 100 articles per search. Set maxItemsPerQuery above 100 with a window of 7 days or longer (or a custom range) and the actor searches day by day, newest first, until the cap is reached: a 7-day window for a busy term returns 500+ articles.

Full article text

{
  "queries": ["tesla earnings"],
  "timeRange": "7d",
  "includeArticleContent": true,
  "maxItems": 50
}

Each article page is fetched and the readable body is extracted (Mozilla Readability, with Article JSON-LD as a fallback), plus author, excerpt, imageUrl, siteName and canonicalUrl. Articles behind a paywall or a bot wall come back with contentExtracted: false and are not charged for content. In a test of 5 articles, 4 returned text (CNBC, Business Insider, Yahoo Finance, KTVU) and Forbes returned HTTP 403.

Monitor mode (alerts)

{
  "queries": ["Acme Corp", "\"Acme\" CEO"],
  "timeRange": "1d",
  "onlyNew": true,
  "monitorId": "acme-press",
  "firstRunBaseline": true,
  "webhookUrl": "https://hooks.slack.com/services/…"
}

The actor remembers every article it has seen for the monitorId (a named key-value store, pruned after 60 days) and returns only new ones, so a schedule produces an alert feed instead of repeats. With firstRunBaseline the first run records what exists today and returns nothing; without it the first run returns the current articles. When new articles are found and webhookUrl is set, a JSON payload (monitorId, newArticles, articles[], runId) is POSTed to it. The run's OUTPUT record in the key-value store summarises every run (articles found and returned per feed, decode failures, monitor state).

All inputs

Field	Type	Default	Description
queries	array	—	search queries (also accepts topic names and Google News URLs)
listUrl	string	—	CSV/TXT/Google Sheet link with queries or URLs in the first column
startUrls	array	—	Google News page URLs (search, topic, section, local, story, rss)
topics	array	—	TOP, WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH
publications	array	—	publisher domains (site: filter)
excludeWords	array	—	words to exclude (-word)
language / country	string	en / US	Google News edition
timeRange	string	any	1h, 1d, 7d, 30d, 1y, custom
dateFrom / dateTo	string	—	YYYY-MM-DD, with timeRange: "custom"
publishedAfter	string	—	ISO date-time; exact lower bound
sortBy	string	relevance	date = newest first per query
maxItems	integer	10	total rows
maxItemsPerQuery	integer	100	rows per query/topic/URL; above 100 triggers day-by-day search; 0 = no cap
resolveUrls	boolean	true	decode Google links to publisher URLs (free)
includeArticleContent	boolean	false	fetch article text, author, image (paid add-on per article with text)
dedupeByUrl	boolean	true	one row per article across all inputs
onlyNew / monitorId / firstRunBaseline / webhookUrl		false / default / false / —	monitor mode
searchQueries / topic			version-1 names, still accepted
proxyConfiguration	object	—	optional; feeds and decoding work without a proxy, and the actor switches to Apify Proxy by itself if Google rate-limits the decoder

Inputs written for other Google News actors run here unchanged: keywords, query, topicUrls, topicsHashed, startInputs, maxArticles, maxArticlesPerInput, timeframe (1h…all, last_hour…last_year), openEndedDateRange (12h, 30d), region_language/regionLanguage (US:en), since/until, siteFilter, decodeUrls, fetchArticleDetails, enrichBody, extractFullText, extractDescriptions.

Output example

A row from a real run with includeArticleContent: true (content shortened):

{
  "title": "Tesla earnings loom after Musk calls SpaceX merger a 'good idea'",
  "url": "https://www.cnbc.com/2026/10/21/tesla-tsla-q3-2026-earnings-report.html",
  "urlResolved": true,
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMie0FVX3lxTE1rRTRZdEQyQ0tYemI0R0tuYi04TWhxWEVoRGx6M3h3MWpWTzhRdnpvZTk4R2tSV09GX2dNS0p6ckxOMkpDd3NCbEViY1ZtOUNOU0JjejU5VjhoVTRIRjZ2SHBiOG0wbXY0NmVBUmFUTDFWZ01LSmZraHN4TdIBgAFBVV95cUxPdzZSSklMOG1fb1hCcUpGSmpyVDAxVlZuXzZ4ZDEwdkJhX2xoWXdVb1V5Vkp0b0tkekhaVHhoa2szSC0tejZ6aTYtb1JJUWo4aG90ZTN5UURiS19sMGhCUHFxZUdhdk9rd3JaN0xRVVk3OGZ0VkhZSjFDMU11dUJIVw?oc=5",
  "guid": "CBMie0FVX3lxTE1rRTRZdEQyQ0tYemI0R0tuYi04TWhxWEVoRGx6M3h3MWpWTzhRdnpvZTk4R2tSV09GX2dNS0p6ckxOMkpDd3NCbEViY1ZtOUNOU0JjejU5VjhoVTRIRjZ2SHBiOG0wbXY0NmVBUmFUTDFWZ01LSmZraHN4TdIBgAFBVV95cUxPdzZSSklMOG1fb1hCcUpGSmpyVDAxVlZuXzZ4ZDEwdkJhX2xoWXdVb1V5Vkp0b0tkekhaVHhoa2szSC0tejZ6aTYtb1JJUWo4aG90ZTN5UURiS19sMGhCUHFxZUdhdk9rd3JaN0xRVVk3OGZ0VkhZSjFDMU11dUJIVw",
  "source": "CNBC",
  "sourceUrl": "https://www.cnbc.com",
  "sourceDomain": "cnbc.com",
  "publishedAt": "2026-10-21T16:00:01.000Z",
  "snippet": null,
  "imageUrl": "https://image.cnbcfm.com/api/v1/image/108307357-1778815414085-gettyimages-2275575381-20090101260514-99-424849.jpeg?v=1779312649&w=1920&h=1080",
  "query": "tesla earnings",
  "topic": null,
  "inputKind": "search",
  "feedTitle": "\"tesla earnings when:7d\" - Google News",
  "language": "en",
  "country": "US",
  "isNew": null,
  "content": "On SpaceX's earnings call late Tuesday, CEO Elon Musk, who also runs Tesla, took a break from talking about financial results to tell investors that it would be a \"good idea\" to merge his two companies …",
  "wordCount": 728,
  "author": "Lora Kolodny",
  "excerpt": "A day after Elon Musk's SpaceX reported third-quarter results, Tesla is scheduled to update investors.",
  "articleTitle": "Tesla earnings loom after Musk calls SpaceX merger a 'good idea'",
  "siteName": "CNBC",
  "canonicalUrl": "https://www.cnbc.com/2026/10/21/tesla-tsla-q3-2026-earnings-report.html",
  "contentExtracted": true,
  "contentError": null,
  "sourceName": "CNBC",
  "publishedDate": "Wed, 21 Oct 2026 16:00:01 GMT",
  "publishedDateISO": "2026-10-21T16:00:01.000Z",
  "description": "Tesla earnings loom after Musk calls SpaceX merger a 'good idea'  CNBC",
  "searchQuery": "tesla earnings",
  "scrapedAt": "2026-10-21T16:16:04.045Z"
}

Pricing

Pay per event, no subscription. See the Pricing tab for your plan's exact figures.

Event	Price	When
Actor start	$0.005	once per run (per GB of memory; the default 512 MB is one event)
Article	$0.0045	every row in the dataset
Article content	$0.002	only with includeArticleContent, only for rows where text was extracted

What that means: 1,000 headlines with real URLs cost $4.50; 1,000 articles with full text cost $6.50. Decoding Google's links, time windows, topics, bulk import, dedupe and monitor mode are included. Paywalled articles that return no text are not charged for content. Set a maximum charge per run in the run options to cap spending; the actor stops at the cap.

Limits and FAQ

How many articles per query? Google serves about 100 per search feed. With a time window of 7 days or more and maxItemsPerQuery above 100, the actor searches one day at a time, newest first, and keeps going until your cap. Topic feeds return 40–70 current stories and cannot be paged.

Why are results not in date order? Google ranks search feeds by relevance. Use sortBy: "date" to get newest first.

Are the URLs real article links? Yes by default. Google News only exposes redirect links (news.google.com/rss/articles/…); the actor decodes each one with the same call the Google News web app makes, at no extra charge. If Google rate-limits that call the actor retries through Apify Proxy; the few that still fail keep the Google link with urlResolved: false.

Why is content empty for some articles? The publisher blocks automated requests or paywalls the page (Forbes, Reuters, Bloomberg, WSJ, FT…). The row says contentExtracted: false with the reason and you are not charged the content event. The actor does not bypass paywalls.

Does the snippet contain the article's first paragraph? No. Google News search feeds have no real snippets (the "description" repeats the headline), so snippet is null there; topic feeds carry the related-coverage headlines. For text use includeArticleContent.

Which countries and languages? Any Google News edition: the country/language pair becomes Google's gl/hl/ceid parameters. Examples that return results: US:en, GB:en, IN:en, AU:en, DE:de, FR:fr, ES:es, MX:es-419, BR:pt-BR, IT:it, NL:nl, PL:pl, JP:ja, KR:ko, TW:zh-Hant, AE:ar.

How do I dedupe across runs? Use guid (or url). Within one run dedupeByUrl already removes repeats; across scheduled runs, monitor mode does it for you.

Images without fetching articles? The feeds carry no images, so imageUrl needs includeArticleContent.

Is this allowed? The actor reads Google News' public feeds and the publishers' public pages, the same data any RSS reader or browser receives. You are responsible for how you use the content; respect publishers' terms and copyright.

Integrations

• API: POST https://api.apify.com/v2/acts/fortuitous_pirate~google-news-scraper/runs?token=… with the input JSON; read …/runs/last/dataset/items?format=json|csv|xlsx.
• Python: ApifyClient(token).actor("fortuitous_pirate/google-news-scraper").call(run_input={"queries": ["OpenAI"], "timeRange": "1d"}), then client.dataset(run["defaultDatasetId"]).iterate_items().
• JavaScript: new ApifyClient({ token }).actor('fortuitous_pirate/google-news-scraper').call({ queries: ['OpenAI'] }).
• Schedules + monitor mode: create a task with onlyNew: true, schedule it hourly or daily, and use webhookUrl or an Apify integration (Slack, Google Sheets, Make, Zapier, n8n, webhooks) to deliver the new articles.
• RSS export: …/actor-tasks/{TASK_ID}/runs/last/dataset/items?format=rss&fields=title,url,snippet,publishedAt&outputFields=title,link,description,pubDate&token=….
• MCP / AI agents: add https://mcp.apify.com?tools=fortuitous_pirate/google-news-scraper to Claude, Cursor or any MCP client and ask for "the latest news about X from the past day".

Changelog

• 2.0 (2026-10-21): real article URLs by default, full article text add-on, startUrls for any Google News page, multi-topic, publishers filter, exclude words, time ranges with past date ranges and day-by-day paging past 100 results, newest-first sort, CSV/Sheet import, dedupe, monitor mode with webhook, compatibility with other actors' input names. Version-1 inputs and output fields unchanged.
• 1.0: keyword and topic search over Google News feeds.