Google News Scraper — Articles & Headlines

Google News Scraper — Articles, Topics & Publisher URLs

Scrape Google News into clean JSON, CSV or Excel: articles for any keyword search (with Google News operators), topics (Top stories, World, Business, Technology, Sports, ...), full coverage story clusters, publications and local news, in any country edition (en-US, en-GB, de-DE, fr-FR, es-MX, pt-BR, ja-JP, ...). Every article comes with the real publisher URL — not the news.google.com/rss/articles/CBMi... redirect — plus source, publish date and, where Google shows them, thumbnail, authors and the story cluster. Optionally the actor opens each article and adds the full text, author, description and canonical URL. No login, no browser, no API key.

What can this Google News scraper do?

• Search Google News for any keyword. Operators work: "exact phrase", -exclude, OR, site:reuters.com, intitle:nvidia, when:1d, when:7d, after:2026-09-01 before:2026-09-08.
• More than 100 results per search: Google returns at most 100 articles per search; above that the actor splits the search into one-day windows (newest first) and collects up to thousands of articles per query.
• Decode publisher URLs: Google News links are opaque redirect ids. The actor resolves them to the original article URL (100% of 700+ articles in our tests).
• Topics and sections per edition: Top stories, World, Nation, Business, Technology, Entertainment, Sports, Science, Health — with thumbnails, authors and story clusters (lead article + related coverage).
• Full coverage stories (news.google.com/stories/...), publications (/publications/...), local news (/headlines/section/geo/...) and RSS feed URLs as input (topic, section, local and search feeds are read from the matching web page or search).
• Any edition / language: one run can scrape the same queries in several countries.
• Article details (optional): canonical URL, authors, description (snippet), full text, publish and modify time from the page metadata, image, keywords, paywall flag.
• Monitoring mode: schedule it and get only articles that are new since the last run — known articles are not charged.

How to scrape Google News

1. Add Search queries (one per line), pick Topics, or paste Google News URLs.
2. Set Editions (e.g. en-US, de-DE, es-MX); empty = en-US.
3. Optionally restrict searches to a Time range (past hour/day/7 days/30 days/year) or From/To dates.
4. Set Max articles per query, topic or URL and Max articles in total, then click Start.

Input example

{
  "searchQueries": ["artificial intelligence", "site:reuters.com tariffs", "intitle:nvidia when:7d"],
  "topics": ["TOP_STORIES", "TECHNOLOGY"],
  "editions": ["en-US", "de-DE"],
  "startUrls": [{ "url": "https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2l4ckplU0VoRUsxTVJlNy1FN2dpZ0FQAQ?hl=en-US&gl=US&ceid=US:en" }],
  "timeRange": "7d",
  "maxArticlesPerSource": 100,
  "maxItems": 1000,
  "includeDetails": false
}

Output example (topic page article)

{
  "title": "Anthropic AI Model Goes Rogue, Submits Fake Unsolved Murder Tip",
  "url": "https://www.wsj.com/us-news/anthropic-ai-model-goes-rogue-submits-fake-unsolved-murder-tip-b0566f54",
  "source": "WSJ",
  "sourceUrl": "https://www.wsj.com",
  "publishedAt": "2026-10-10T03:34:00Z",
  "thumbnail": "https://images.wsj.net/im-34410218?width=700&height=466",
  "authors": ["Joseph Pisani", "Georgia W."],
  "storyId": "CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lleElPVEVoRkZnbDFTZXRTSVh5Z0FQAQ",
  "storyUrl": "https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lleElPVEVoRkZnbDFTZXRTSVh5Z0FQAQ",
  "isStoryLead": true,
  "relatedArticles": [
    {"id": "CBMi...", "title": "Anthropic AI model submits false tip on unsolved Philly murder", "source": "NBC10 Philadelphia", "url": "https://www.nbcphiladelphia.com/news/..."}
  ],
  "id": "CBMioAFBVV95cUxOdC1Yb1VaRzVm...",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMioAFBVV95cUxOdC1Yb1VaRzVm...?oc=5",
  "urlDecoded": true,
  "publisherUrlResolved": true,
  "ampUrl": null,
  "sourceType": "topic",
  "query": null,
  "topic": "BUSINESS",
  "inputUrl": null,
  "edition": "en-US",
  "ceid": "US:en",
  "position": 1,
  "scrapedAt": "2026-10-10T05:06:03Z"
}

Output example (search result with article details)

{
  "title": "Why Nvidia (NVDA) Stock Is Trading Lower Today",
  "url": "https://finance.yahoo.com/markets/stocks/articles/why-nvidia-nvda-stock-trading-000009960.html",
  "source": "Yahoo Finance",
  "sourceUrl": "https://finance.yahoo.com",
  "publishedAt": "2026-10-09T00:00:00Z",
  "thumbnail": "https://s.yimg.com/.../c941ddf65a337c10e3f4bc08b2a8e4cb.jpg",
  "authors": ["Jabin Bastian"],
  "storyId": null,
  "sourceType": "search",
  "query": "nvidia",
  "edition": "en-US",
  "urlDecoded": true,
  "publisherUrlResolved": true,
  "canonicalUrl": "https://finance.yahoo.com/markets/stocks/articles/why-nvidia-nvda-stock-trading-000009960.html",
  "description": "Shares of leading AI chipmaker Nvidia fell 2.1% in the afternoon session ...",
  "text": "Shares of leading AI chipmaker Nvidia (NASDAQ:NVDA) fell 2.1% in the afternoon session ...",
  "wordCount": 433,
  "publishedAtMeta": "2026-10-09T00:00:09Z",
  "modifiedAt": "2026-10-09T00:00:09Z",
  "siteName": "Yahoo Finance",
  "language": "en-US",
  "keywords": [],
  "isPaywalled": null,
  "detailError": null
}

Fields

Field	Description
title	Headline as Google News shows it (the trailing " - Publisher" is removed)
url	The real publisher article URL (resolved from Google's redirect link)
source, sourceUrl	Publisher name and homepage
publishedAt	Publish time Google News reports (UTC, ISO 8601)
thumbnail	Article image (topic, home and story pages; search results get it with article details)
authors	Bylines (topic/story pages, or from the article page with details)
storyId, storyUrl	Google's "Full coverage" story cluster the article belongs to
isStoryLead, relatedArticles[]	Cluster lead and the other coverage Google groups with it (id, title, source, url)
id, googleNewsUrl	Google's article id and link (stable; used for monitoring). Topic/story/home articles: the Google News id (CBMi...) and news.google.com link; search results: Google Search's news document id and its google.com/goto redirect link
urlDecoded	true when url is the publisher URL; false if decoding failed (then url = Google link)
publisherUrlResolved	true when url is the real publisher URL, false when it is still Google's link. Google's search-result links for some articles cannot be turned into the publisher URL by any route we tested; those keep the Google link and publisherUrlResolved: false. Topic and RSS articles always carry the publisher URL
ampUrl	AMP version when Google lists one
sourceType, query, topic, inputUrl	Where the article came from (search, topic, story, publication, geo, home, rss)
edition, ceid, position	Edition (language-country), Google's ceid and the rank on the page or feed
canonicalUrl, description, text, wordCount	Article details: canonical link, meta description (snippet), extracted full text
publishedAtMeta, modifiedAt	Publish/modify time from the article page metadata (JSON-LD / meta tags)
siteName, language, keywords, isPaywalled	More page metadata
detailError	Why the article page could not be read (e.g. HTTP 401 from a bot-protected site)
changeType	Monitoring mode only: new

How to get more than 100 Google News results for a search

Google returns at most about 100 articles per search (10 pages of 10). Set Max articles per query, topic or URL above 100 and the actor splits the search into one-day windows with Google's after:/before: operators, newest first, from To date (or today) back to From date (or the start of the Time range, or up to about a year). Each day can again return up to 100 articles, so a busy topic yields thousands of articles (each page of 10 is one Google request, so very large collections take a while). Duplicates are removed. A query that already contains when:, after: or before: is not split.

How are Google News redirect links decoded?

Search results link to Google redirects (google.com/goto?url=...); the actor reads where each redirect points (without opening the publisher page) and returns the original URL in url; the Google link stays in googleNewsUrl. Google News article links (news.google.com/.../articles/CBMi...) no longer contain the publisher URL since 2024; the actor resolves them the way the Google News website does (a signed lookup, many articles per request). Articles from topic, home and full coverage pages already carry the real URL, so they need no decoding. Turn Decode publisher URLs off if you only need titles, sources and dates (faster).

Monitor changes on a schedule

Turn on Only new or changed items and save the input as a task with a Schedule (Apify Console: Saved tasks -> Schedule, for example every hour). Each run remembers which articles earlier runs of the same input returned and outputs only articles that are new, so you get a news alert feed for your keywords or topics. Known articles are skipped before any URL decoding, so they cost neither time nor money. New articles carry changeType: "new" (previousValues is reserved for changed items and does not occur for articles); use Monitoring key to keep several schedules on the same input apart.

How much does it cost to scrape Google News?

$0.003 per article (article event: real publisher URL, source, date and everything Google shows), or $0.004 per article with page details (article_details: plus author, description, full text and page metadata), Apify platform usage (compute, proxy, storage) is included in these prices. No start fee. 1,000 search results with publisher URLs cost $3.00 in total.

Does it work without login or a Google API key?

Yes. Google has no official Google News API; the actor reads the public web pages of news.google.com (topics, sections, full coverage stories) and, for keyword searches, the News tab of Google Search through Apify's Google SERP proxy, with no account, no browser and no CAPTCHA solving. Since October 2026 Google answers the news.google.com RSS feeds with "automated queries" errors, so searches no longer use them.

Why do some articles have no thumbnail, author or full text?

Search results come from the News tab of Google Search, which shows no bylines and only tiny inline thumbnails: switch on Fetch article pages to add them from the publisher page (thumbnail from its preview image; description falls back to Google's snippet). Topic, home and full coverage pages include them directly. Some publishers (for example Reuters, WSJ, Bloomberg, Barron's) block automated visitors or hide articles behind a paywall; their articles still come with title, URL, source and date, and detailError tells you why the page could not be read.

Proxy note

Keyword searches always go through Apify's Google SERP proxy (included in the price). Google News requests (topic pages and URL lookups) go out directly from the Apify server first: that is about 10 times faster than sending the 600 KB lookup pages through the datacenter proxy. If Google throttles that route (HTTP 429, "unusual traffic" page, timeouts), the actor switches to your proxy configuration for the rest of the run, with its own cookies per proxy IP and throttled IPs replaced; if the datacenter IPs keep getting throttled too, it moves to the RESIDENTIAL group when your plan includes it. Publisher article pages (Fetch article pages) always use your proxy configuration.

Is it legal to scrape Google News?

The actor collects publicly available news headlines, links and metadata that Google News shows to every visitor without an account, and (optionally) publicly accessible article pages. It does not log in, bypass paywalls or solve CAPTCHAs. Article texts are copyrighted by their publishers: use them in line with copyright law and the publishers' terms (for example for analysis, monitoring or with a license). You are responsible for how you use the data.

Support

Found a bug or need a field? Open an issue in the Issues tab; it is usually answered within a day.