# Google News Scraper — Articles, Topics & Publisher URLs (`tortuga/google-news-scraper`) Actor Scrape Google News searches, topics, full coverage stories and publications in any country edition: real publisher URLs, source, date, thumbnail, authors; optional full text. - **URL**: https://apify.com/tortuga/google-news-scraper.md - **Developed by:** [Trevor Ortega](https://apify.com/tortuga) (community) - **Categories:** News - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 articles This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Google News Scraper — Articles, Topics & Publisher URLs Scrape **Google News** into clean JSON, CSV or Excel: articles for any **keyword search** (with Google News operators), **topics** (Top stories, World, Business, Technology, Sports, ...), **full coverage story clusters**, **publications** and local news, in **any country edition** (en-US, en-GB, de-DE, fr-FR, es-MX, pt-BR, ja-JP, ...). Every article comes with the **real publisher URL** — not the `news.google.com/rss/articles/CBMi...` redirect — plus source, publish date and, where Google shows them, thumbnail, authors and the story cluster. Optionally the actor opens each article and adds the **full text**, author, description and canonical URL. No login, no browser, no API key. ### What can this Google News scraper do? - **Search Google News** for any keyword. Operators work: `"exact phrase"`, `-exclude`, `OR`, `site:reuters.com`, `intitle:nvidia`, `when:1d`, `when:7d`, `after:2026-09-01 before:2026-09-08`. - **More than 100 results per search**: Google returns at most 100 articles per search; above that the actor splits the search into one-day windows (newest first) and collects up to thousands of articles per query. - **Decode publisher URLs**: Google News links are opaque redirect ids. The actor resolves them to the original article URL (100% of 700+ articles in our tests). - **Topics and sections** per edition: Top stories, World, Nation, Business, Technology, Entertainment, Sports, Science, Health — with thumbnails, authors and story clusters (lead article + related coverage). - **Full coverage** stories (`news.google.com/stories/...`), **publications** (`/publications/...`), **local news** (`/headlines/section/geo/...`) and RSS feed URLs as input. - **Any edition / language**: one run can scrape the same queries in several countries. - **Article details (optional)**: canonical URL, authors, description (snippet), full text, publish and modify time from the page metadata, image, keywords, paywall flag. - **Monitoring mode**: schedule it and get only articles that are new since the last run — known articles are not charged. ### How to scrape Google News 1. Add **Search queries** (one per line), pick **Topics**, or paste **Google News URLs**. 2. Set **Editions** (e.g. `en-US`, `de-DE`, `es-MX`); empty = `en-US`. 3. Optionally restrict searches to a **Time range** (past hour/day/7 days/30 days/year) or **From/To dates**. 4. Set **Max articles per query, topic or URL** and **Max articles in total**, then click **Start**. #### Input example ```json { "searchQueries": ["artificial intelligence", "site:reuters.com tariffs", "intitle:nvidia when:7d"], "topics": ["TOP_STORIES", "TECHNOLOGY"], "editions": ["en-US", "de-DE"], "startUrls": [{ "url": "https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2l4ckplU0VoRUsxTVJlNy1FN2dpZ0FQAQ?hl=en-US&gl=US&ceid=US:en" }], "timeRange": "7d", "maxArticlesPerSource": 100, "maxItems": 1000, "includeDetails": false } ``` #### Output example (topic page article) ```json { "title": "Anthropic AI Model Goes Rogue, Submits Fake Unsolved Murder Tip", "url": "https://www.wsj.com/us-news/anthropic-ai-model-goes-rogue-submits-fake-unsolved-murder-tip-b0566f54", "source": "WSJ", "sourceUrl": "https://www.wsj.com", "publishedAt": "2026-10-10T03:34:00Z", "thumbnail": "https://images.wsj.net/im-34410218?width=700&height=466", "authors": ["Joseph Pisani", "Georgia W."], "storyId": "CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lleElPVEVoRkZnbDFTZXRTSVh5Z0FQAQ", "storyUrl": "https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lleElPVEVoRkZnbDFTZXRTSVh5Z0FQAQ", "isStoryLead": true, "relatedArticles": [ {"id": "CBMi...", "title": "Anthropic AI model submits false tip on unsolved Philly murder", "source": "NBC10 Philadelphia", "url": "https://www.nbcphiladelphia.com/news/..."} ], "id": "CBMioAFBVV95cUxOdC1Yb1VaRzVm...", "googleNewsUrl": "https://news.google.com/rss/articles/CBMioAFBVV95cUxOdC1Yb1VaRzVm...?oc=5", "urlDecoded": true, "ampUrl": null, "sourceType": "topic", "query": null, "topic": "BUSINESS", "inputUrl": null, "edition": "en-US", "ceid": "US:en", "position": 1, "scrapedAt": "2026-10-10T05:06:03Z" } ``` #### Output example (search result with article details) ```json { "title": "Why Nvidia (NVDA) Stock Is Trading Lower Today", "url": "https://finance.yahoo.com/markets/stocks/articles/why-nvidia-nvda-stock-trading-000009960.html", "source": "Yahoo Finance", "sourceUrl": "https://finance.yahoo.com", "publishedAt": "2026-10-09T00:00:00Z", "thumbnail": "https://s.yimg.com/.../c941ddf65a337c10e3f4bc08b2a8e4cb.jpg", "authors": ["Jabin Bastian"], "storyId": null, "sourceType": "search", "query": "nvidia", "edition": "en-US", "urlDecoded": true, "canonicalUrl": "https://finance.yahoo.com/markets/stocks/articles/why-nvidia-nvda-stock-trading-000009960.html", "description": "Shares of leading AI chipmaker Nvidia fell 2.1% in the afternoon session ...", "text": "Shares of leading AI chipmaker Nvidia (NASDAQ:NVDA) fell 2.1% in the afternoon session ...", "wordCount": 433, "publishedAtMeta": "2026-10-09T00:00:09Z", "modifiedAt": "2026-10-09T00:00:09Z", "siteName": "Yahoo Finance", "language": "en-US", "keywords": [], "isPaywalled": null, "detailError": null } ``` #### Fields | Field | Description | |---|---| | `title` | Headline as Google News shows it (the trailing " - Publisher" is removed) | | `url` | The real publisher article URL (decoded from Google's redirect id) | | `source`, `sourceUrl` | Publisher name and homepage | | `publishedAt` | Publish time Google News reports (UTC, ISO 8601) | | `thumbnail` | Article image (topic, home and story pages; search results get it with article details) | | `authors` | Bylines (topic/story pages, or from the article page with details) | | `storyId`, `storyUrl` | Google's "Full coverage" story cluster the article belongs to | | `isStoryLead`, `relatedArticles[]` | Cluster lead and the other coverage Google groups with it (`id`, `title`, `source`, `url`) | | `id`, `googleNewsUrl` | Google News article id and link (stable; used for monitoring) | | `urlDecoded` | `true` when `url` is the publisher URL; `false` if decoding failed (then `url` = Google link) | | `ampUrl` | AMP version when Google lists one | | `sourceType`, `query`, `topic`, `inputUrl` | Where the article came from (`search`, `topic`, `story`, `publication`, `geo`, `home`, `rss`) | | `edition`, `ceid`, `position` | Edition (language-country), Google's ceid and the rank on the page or feed | | `canonicalUrl`, `description`, `text`, `wordCount` | Article details: canonical link, meta description (snippet), extracted full text | | `publishedAtMeta`, `modifiedAt` | Publish/modify time from the article page metadata (JSON-LD / meta tags) | | `siteName`, `language`, `keywords`, `isPaywalled` | More page metadata | | `detailError` | Why the article page could not be read (e.g. HTTP 401 from a bot-protected site) | | `changeType` | Monitoring mode only: `new` | ### How to get more than 100 Google News results for a search Google News returns at most 100 articles per search. Set **Max articles per query, topic or URL** above 100 and the actor splits the search into one-day windows with Google's `after:`/`before:` operators, newest first, from **To date** (or today) back to **From date** (or the start of the **Time range**, or up to about a year). Each day can again return up to 100 articles, so a busy topic yields thousands of articles. Duplicates are removed. A query that already contains `when:`, `after:` or `before:` is not split. ### How are Google News redirect links decoded? Since 2024 Google News links (`news.google.com/rss/articles/CBMi...`) no longer contain the publisher URL. The actor resolves each one the way the Google News website does (a signed lookup, many articles per request) and returns the original URL in `url`; the Google link stays in `googleNewsUrl`. Articles from topic, home and full coverage pages already carry the real URL, so they need no decoding. Turn **Decode publisher URLs** off if you only need titles, sources and dates (faster). ### Monitor changes on a schedule Turn on **Only new or changed items** and save the input as a task with a Schedule (Apify Console: **Saved tasks** -> **Schedule**, for example every hour). Each run remembers which articles earlier runs of the same input returned and outputs only articles that are new, so you get a news alert feed for your keywords or topics. Known articles are skipped before any URL decoding, so they cost neither time nor money. New articles carry `changeType: "new"` (`previousValues` is reserved for changed items and does not occur for articles); use **Monitoring key** to keep several schedules on the same input apart. ### How much does it cost to scrape Google News? **$0.003 per article** (`article` event: real publisher URL, source, date and everything Google shows), or **$0.004 per article with page details** (`article_details`: plus author, description, full text and page metadata), plus Apify platform usage paid by you: about **$0.007 per 1,000 articles** on the default 512 MB memory, most of it Apify's dataset storage fee (1,300 articles incl. 800 decoded search results ran in 75 seconds for $0.009), and about $0.04 per 1,000 with article pages. No start fee. 1,000 search results with publisher URLs cost about $3.01 in total. ### Does it work without login or a Google API key? Yes. Google has no official Google News API; the actor reads the public RSS feeds and web pages of news.google.com with no account, no browser and no CAPTCHA solving. Google throttles busy IPs, so the actor spreads requests over fresh IPs with their own cookies when needed (see the proxy note); in our tests 100% of feeds (30 of 30), web pages and URL decodings (1,800+ of 1,800+) succeeded. ### Why do some articles have no thumbnail, author or full text? Search results come from Google's RSS feed, which has no images or bylines: switch on **Fetch article pages** to add them from the publisher page (`thumbnail` from its preview image). Topic, home and full coverage pages include them directly. Some publishers (for example Reuters, WSJ, Bloomberg, Barron's) block automated visitors or hide articles behind a paywall; their articles still come with title, URL, source and date, and `detailError` tells you why the page could not be read. ### Proxy note Google News requests (feeds, topic pages and URL lookups) go out directly from the Apify server first: that is about 10 times faster than sending the 600 KB lookup pages through the datacenter proxy. If Google throttles that route (HTTP 429, "unusual traffic" page, timeouts), the actor switches to your proxy configuration for the rest of the run, with its own cookies per proxy IP and throttled IPs replaced; if the datacenter IPs keep getting throttled too, it moves to the RESIDENTIAL group when your plan includes it. Publisher article pages (**Fetch article pages**) always use your proxy configuration. ### Is it legal to scrape Google News? The actor collects publicly available news headlines, links and metadata that Google News shows to every visitor without an account, and (optionally) publicly accessible article pages. It does not log in, bypass paywalls or solve CAPTCHAs. Article texts are copyrighted by their publishers: use them in line with copyright law and the publishers' terms (for example for analysis, monitoring or with a license). You are responsible for how you use the data. ### Support Found a bug or need a field? Open an issue in the **Issues** tab; it is usually answered within a day. # Actor input Schema ## `searchQueries` (type: `array`): Keywords to search on Google News, one per line. Google News operators work: "exact phrase", -exclude, OR, site:reuters.com, intitle:nvidia, when:1d / when:7d, after:2026-09-01 before:2026-09-08. ## `topics` (type: `array`): Google News sections to scrape for each edition: Top stories, World, the edition's country (Nation), Business, Technology, Entertainment, Sports, Science, Health. Topic pages give real publisher URLs, thumbnails, authors and story clusters directly. ## `startUrls` (type: `array`): Optional: paste news.google.com URLs. Supported: search (/search?q=), topics (/topics/...), sections (/headlines/section/topic/BUSINESS), local news (/headlines/section/geo/...), full coverage stories (/stories/...), publications (/publications/...), the home page and RSS feed URLs. The URL's hl/gl/ceid edition is kept. ## `editions` (type: `array`): Google News editions to scrape each query and topic in, as language-COUNTRY: en-US, en-GB, en-IN, de-DE, fr-FR, es-ES, es-MX, pt-BR, it-IT, ja-JP, ... A ceid such as US:en also works. Empty = en-US. ## `timeRange` (type: `string`): Only articles from the past hour, day, 7 days, 30 days or year. Adds Google's when: operator to queries that have no date operator of their own. ## `dateFrom` (type: `string`): Only articles published on or after this date: YYYY-MM-DD or a period such as '30 days'. Uses Google's after: operator. ## `dateTo` (type: `string`): Only articles published on or before this date (YYYY-MM-DD). Uses Google's before: operator. ## `maxArticlesPerSource` (type: `integer`): Per search query, topic or URL in each edition. Google returns up to 100 articles per search; above 100 the search is split into one-day windows (newest first, back from 'To date' or today) to collect more. ## `decodeUrls` (type: `boolean`): Resolve Google's news.google.com/rss/articles/... redirect links of search results to the real publisher article URL (2 light requests per article). Turn off to only get titles, sources and dates faster. Topic and story pages always have the real URL. ## `includeDetails` (type: `boolean`): Open every publisher article page and add canonical URL, authors, description (snippet), full text (readability-style extraction), publish/modify time from the page metadata, image, keywords and paywall flag. One extra request per article; charged as 'article with details'. Paywalled or bot-protected sites return metadata only or a detailError. ## `includeFullText` (type: `boolean`): With article pages on: keep the extracted article text. Turn off to keep only metadata (smaller dataset). ## `maxItems` (type: `integer`): Stop after this many articles for the whole run. Keeps cost predictable. ## `onlyNewOrChanged` (type: `boolean`): Remember what earlier runs of this search returned and output only new items or items whose price/status changed. Use with a Schedule; unchanged items are not charged. Here: each article is output once, the first time it appears for this input; known articles are skipped before any URL decoding. ## `monitoringKey` (type: `string`): Name for this watch, so several schedules on the same search stay separate; defaults to a key derived from the search input. ## `proxyConfiguration` (type: `object`): Used for publisher article pages, and for Google News when Google throttles the direct route (Google requests go out directly first: ~10x faster). Each proxy IP gets its own Google cookies; throttled IPs are replaced, and RESIDENTIAL is tried when datacenter IPs stay throttled and your plan has it. ## Actor input object example ```json { "searchQueries": [ "artificial intelligence" ], "editions": [ "en-US" ], "maxArticlesPerSource": 100, "decodeUrls": true, "includeDetails": false, "includeFullText": true, "maxItems": 100, "onlyNewOrChanged": false, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQueries": [ "artificial intelligence" ], "editions": [ "en-US" ], "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("tortuga/google-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQueries": ["artificial intelligence"], "editions": ["en-US"], "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("tortuga/google-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQueries": [ "artificial intelligence" ], "editions": [ "en-US" ], "proxyConfiguration": { "useApifyProxy": true } }' | apify call tortuga/google-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,tortuga/google-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/vZSoEEySd8qgcFRQI/builds/oJDW9snJiSGAPaVoY/openapi.json