# Google News Scraper API - Full Text, Monitoring & RSS (`automly/google-news-scraper-api`) Actor Scrape Google News by keyword and get real publisher URLs, full article text and only new articles on every scheduled run. Adds the GDELT global news index and any RSS or Atom feed, deduplicated into one table. No API key needed. For media monitoring, brand tracking and news datasets. - **URL**: https://apify.com/automly/google-news-scraper-api.md - **Developed by:** [Automly](https://apify.com/automly) (community) - **Categories:** News, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.50 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google News Scraper API — Full Text, Monitoring & RSS Search Google News by keyword and get back the publisher's real link, the full article text, and on repeat runs only the stories you haven't seen yet. No API key, no login. You can fold in the GDELT global news index and any RSS or Atom feed as well, and it all arrives as one table you can export to JSON, CSV or Excel. ### What does Google News Scraper API do? - Searches Google News and gives you the publisher's own link, not a `news.google.com/rss/articles/CBMi…` redirect - Pulls the article body, so you get more than a headline and two lines of summary - Goes past Google's ceiling of roughly 100 articles per query by walking a date range one day at a time - Remembers what it already sent you, so a scheduled run reports only new stories - Filters on publisher domain and on whole words, so `car` never matches `cargo` - Narrows to a language or a country from a dropdown, covering 146 languages and 235 countries - Reads GDELT and any RSS feed into the same table, with BBC, CNN, NPR, New York Times, Guardian, Google News and Hacker News ready to pick from ### What people use it for - Watching for mentions of a brand or a competitor, with one row per story instead of the same piece three times - Media monitoring and PR reporting, where you need the publisher, the date and a link that stays stable - Building research datasets, with the article text across a stretch of dates - Feeding alerts: schedule it, get only what's new, send it to Slack or a webhook ### How to scrape Google News 1. Click **Try for free** to open the Actor in Apify Console 2. Type a keyword into **Search queries**, say `tesla recall` 3. Choose your **Sources**: `google-news`, `gdelt`, `rss`, or all three 4. Add **Preset feeds** (BBC, Guardian, and so on) or paste feed URLs of your own, if you want them 5. Set **Maximum results** and hit **Start** 6. Grab the results from the **Dataset** tab as JSON, CSV or Excel Three switches are worth knowing about. **Include full text** fetches each article and pulls out the body. **Resolve Google News URLs** turns those redirect links into real publisher links, and full text turns it on for you anyway. **Skip already seen articles** is the one that makes scheduled runs useful, because each run then brings back only what the last one missed. ### Use it as a Google News API A single request gives you the rows straight back, so you can treat it as a news endpoint from any language: ```bash curl -X POST "https://api.apify.com/v2/acts/automly~google-news-scraper-api/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \ -H 'Content-Type: application/json' \ -d '{ "queries": ["tesla recall"], "sources": ["google-news", "gdelt"], "presetFeeds": ["bbc", "guardian"], "maxResults": 50, "includeKeywords": ["tesla"], "resolveGoogleUrls": true }' ``` Or from Python, using the `apify-client` package: ```python from apify_client import ApifyClient client = ApifyClient("YOUR_API_TOKEN") run = client.actor("automly/google-news-scraper-api").call( run_input={"queries": ["tesla recall"], "includeFullText": True, "maxResults": 50} ) for article in client.dataset(run["defaultDatasetId"]).iterate_items(): print(article["publishedAt"], article["sourceDomain"], article["title"]) ``` You can also schedule the same run in Apify Console and wire it up to Slack, Google Sheets, webhooks, Make, n8n or Zapier through Apify integrations. ### Input parameters | Parameter | Type | Description | |---|---|---| | `queries` | string\[] | Search terms, one per line, up to 100. Sent to every search source you enabled | | `feedUrls` | string\[] | RSS or Atom feed URLs to read directly, up to 200 | | `presetFeeds` | string\[] | Well-known feeds by name: `bbc`, `cnn`, `npr`, `nyt`, `guardian`, `google-news-top`, `hacker-news` | | `sources` | string\[] | Any of `google-news`, `gdelt`, `rss`. Defaults to `google-news` | | `maxResults` | integer | How many articles to collect in total, 1–50000. Defaults to 100 | | `sinceHours` | integer | Only articles from the last N hours, 1–8760 | | `fromDate` / `toDate` | string | `YYYY-MM-DD` bounds. A date range wins over `sinceHours` | | `includeKeywords` | string\[] | Keep articles whose headline or summary contains one of these. Whole words, case-insensitive. Filtering happens before the body is fetched, so bodies aren't searched | | `excludeKeywords` | string\[] | Drop articles containing any of these. Exclusions beat includes | | `includeDomains` | string\[] | Keep only these domains, like `reuters.com`. Subdomains count, and `www.` and ports are ignored | | `excludeDomains` | string\[] | Drop these domains and anything under them | | `language` | string | Pick a language from the dropdown. It sets the Google News edition and drops rows from sources that report a different language | | `country` | string | Pick a country from the dropdown. It sets the Google News edition and drops rows from sources that report a different country | | `resolveGoogleUrls` | boolean | Turn Google News redirect links into real publisher links | | `includeFullText` | boolean | Fetch each article page and pull out the body. Resolves Google News links for you. Some publishers refuse automated fetches, so a few rows keep their summary and leave the body empty | | `skipSeenArticles` | boolean | Return only articles that earlier runs didn't deliver | | `seenArticlesStoreName` | string | Names the store of delivered articles. Use a separate one per monitored query | | `proxyConfiguration` | object | Proxy settings. Leave a rotating group switched on | You need at least one of `queries`, `feedUrls` or `presetFeeds`. Adding a feed or a preset switches the `rss` source on by itself. ### Output example One row per article, nineteen fields. This is a real row from a run on 12 September 2026: ```json { "title": "Lawmaker Demand Feds Take Action After 43 Tesla Drivers Filmed Asleep At The Wheel", "url": "https://insideevs.com:443/news/807893/tesla-sleeping-congress-nhtsa-demand/", "canonicalUrl": "https://insideevs.com/news/807893/tesla-sleeping-congress-nhtsa-demand", "source": "gdelt", "sourceName": "insideevs.com", "sourceDomain": "insideevs.com", "publishedAt": "2026-09-11T23:15:00+00:00", "summary": null, "author": null, "imageUrl": "https://cdn.motor1.com/images/mgl/W87NGL/s1/tesla-fsd-with-a-dead-battery.jpg", "categories": [], "language": "English", "country": "United States", "query": "tesla recall", "guid": null, "resolved": true, "fullText": null, "fullTextChars": null, "scrapedAt": "2026-09-12T12:52:24.373597+00:00" } ``` `canonicalUrl` is the field deduplication works from. It lowercases the host, drops `www.`, the fragment and a redundant port, and strips the usual tracking parameters such as `utm_*`, `at_*`, `fbclid` and `gclid`. That's why the same story reaching you from two different feeds still counts once. Switch on `resolveGoogleUrls` and a Google News row carries the publisher's own link instead: ```json { "title": "Driver crashes Tesla into scaffolding in midtown Manhattan, killing passenger: Police - ABC News", "url": "https://abcnews.com/US/tesla-crashes-scaffolding-midtown-manhattan-killing-passenger/story?id=136300943", "source": "google-news", "sourceDomain": "abcnews.com", "resolved": true } ``` Switch on `includeFullText` and each row gains the body: ```json { "title": "Tesla and others begin record vehicle recall in China - Reuters", "url": "https://www.reuters.com/world/tesla-fix-software-millions-china-made-imported-evs-china-2026-08", "sourceDomain": "reuters.com", "fullTextChars": 4258, "fullText": "BEIJING, Aug 21 (Reuters) - Tesla and eight other automakers said on Friday they will recall a total of about 4.3 million vehicles in China over concerns that doors may be difficult to open…" } ``` The three sources don't publish the same things, so some columns are fuller than others: | Source | You get | You don't | |---|---|---| | `rss` | headline, link, date, summary, guid, and author, image or categories when the feed bothers to include them | `language`, `country` | | `google-news` | headline, link, date, summary, guid, publisher domain | `author`, `categories`, `language`, `country` | | `gdelt` | headline, link, date, image, `language`, `country` | `summary`, `author`, `guid` | `fullText` and `fullTextChars` only appear when you ask for full text. ### How many Google News articles can you get per query? Google News gives you about 100 articles for a single query, and GDELT stops at 250. To go further, ask for more than that *and* give a date range. The range then gets split into daily windows and queried a day at a time, so a week of history can return several hundred rather than one hundred. Ask for 500 with no date range and you'll still hit the ceiling. When you pick several sources, rows come back round-robin rather than one source at a time. Ask for 60 across three sources and you'll get roughly 20 from each, instead of 60 from whichever happened to be fetched first. ### Monitoring: only new articles on every run Switch on `skipSeenArticles` and every run returns only the articles no earlier run delivered, which is what makes polling on a schedule worthwhile. In testing, a first run stored 60 articles and an identical second run stored 34. Give each query you monitor its own `seenArticlesStoreName` so they don't share a memory. ### What it doesn't do No sentiment scoring, no entity extraction, no translation, no paywall bypass. GDELT also rate-limits by IP and will refuse a request now and then, which shows up in the log as `GDELT returned HTTP 429`. The run retries on a fresh address, and your other sources carry on unaffected. ### FAQ #### Do I need an API key for Google News or GDELT? No. It reads Google News's public RSS feeds and GDELT's public DOC API. Neither asks for credentials, and no login or cookies are involved. #### Can I get more than 100 articles from one Google News query? Yes. Set `maxResults` above the ceiling and give it `fromDate` and `toDate`. The range gets split into daily windows and queried one day at a time, so each day can add up to about another hundred articles. #### Why do Google News links point at google.com instead of the publisher? Because Google News publishes encoded redirect links, and following one lands you back inside Google rather than on the article. Switch on `resolveGoogleUrls` and every link becomes the publisher's own. That also lets the same story be matched against GDELT and RSS copies when duplicates are removed. #### How do I get the full article text and not just the summary? Switch on `includeFullText`. Each article page gets fetched and its main text pulled out — a Reuters piece came back at 4,258 characters — and Google News links are resolved along the way so the body is actually reachable. Expect a few rows to come back without text. On one 80-article run, 71 had text and 9 didn't: those publishers either refused the fetch outright or served a page with no article in the HTML. Those rows keep their summary, and the body is left empty rather than stuffed with navigation text. The run log tells you the split, so you're not left guessing why a column has gaps. #### Is the same article returned twice if two sources carry it? Not once Google News links are resolved. Deduplication works from `canonicalUrl`, so one story counts once across all three sources. The exception is a Google News row you left unresolved: it has no publisher link to compare against, so it's keyed on its Google id and can't be matched with a GDELT or RSS copy of the same piece. Switching on `resolveGoogleUrls` brings those rows in too. #### Can I use it instead of Google Alerts? For keyword monitoring, yes. Schedule it, switch on `skipSeenArticles`, and connect the run to Slack, email or a webhook through Apify integrations. Each run then delivers only the stories earlier runs didn't, with the publisher link, the date and optionally the full text, ready to filter or export. #### Is it legal to scrape Google News? It collects publicly available article metadata from public RSS feeds and GDELT's public API, without logging in, and full text comes from the publisher's own public page. Complying with the terms of the sources you use, with copyright, and with data-protection law where you are is down to you. #### What output formats are supported? JSON, CSV, Excel, XML and JSONL, from the Dataset tab or the API, plus Apify integrations for Google Sheets, Slack, webhooks and S3. # Actor input Schema ## `queries` (type: `array`): Enter one search query per line. Every query is sent to each enabled search source. Provide at least one query, feed URL or preset feed. ## `feedUrls` (type: `array`): Paste RSS or Atom feed URLs to read directly. Adding a feed turns the RSS source on by itself, so you do not have to select it below. ## `presetFeeds` (type: `array`): Pick well-known feeds instead of looking up their URLs. Selecting a preset turns the RSS source on by itself. ## `sources` (type: `array`): Choose which sources to collect from. Google News and GDELT need at least one search query; RSS needs a feed URL or a preset feed. ## `maxResults` (type: `integer`): Set the total number of articles to collect. Google News returns about 100 articles per query and GDELT at most 250, so ask for more than that together with a date range and the actor splits the range into daily windows. ## `sinceHours` (type: `integer`): Collect only articles published in the last N hours. Google News receives this as its own recency operator. The date range below takes precedence when you set both. ## `fromDate` (type: `string`): Collect articles published on or after this day, as YYYY-MM-DD. Pair a wide range with a high maximum results to make the actor split the range into daily windows and get past the per-query caps. ## `toDate` (type: `string`): Collect articles published on or before this day, as YYYY-MM-DD. Leave empty to collect up to now. ## `includeKeywords` (type: `array`): Keep only articles whose headline, summary or extracted body text contains at least one of these words. Matching ignores case and needs a whole word, so "car" does not match "cargo". ## `excludeKeywords` (type: `array`): Drop articles whose headline, summary or extracted body text contains any of these words. Exclusions win over the include list. ## `includeDomains` (type: `array`): Keep only articles from these publisher domains, for example reuters.com. Subdomains are matched too, and a leading www. is ignored. ## `excludeDomains` (type: `array`): Drop articles from these publisher domains and their subdomains. Exclusions win over the include list. ## `language` (type: `string`): Pick the language edition Google News answers in. The same choice also drops articles that report a different language of their own, so a source that reports one - GDELT does, Google News and RSS do not - is filtered to match. Leave it empty for the English edition and no language filter. ## `country` (type: `string`): Pick the country edition Google News answers from. The same choice also drops articles that report a different country of their own, so a source that reports one - GDELT does, Google News and RSS do not - is filtered to match. Leave it empty for the US edition and no country filter. ## `resolveGoogleUrls` (type: `boolean`): Turn Google News redirect links into real publisher URLs. Each link costs about 120 KB of proxy traffic; links that cannot be resolved are left as they are. ## `includeFullText` (type: `boolean`): Download each article page and extract its body text. Google News links are resolved to the publisher automatically when this is on, because a news.google.com link is a redirect page with no article in it. Adds proxy traffic per article. Some publishers refuse automated fetches: on a measured 80-article run, 71 returned text and 9 did not. Those rows keep their summary and leave the body empty rather than storing navigation, and the run log reports the split. ## `skipSeenArticles` (type: `boolean`): Return only articles that earlier runs did not already deliver. Use this to poll a query on a schedule. ## `seenArticlesStoreName` (type: `string`): Name the named key-value store that remembers delivered articles. Give each monitored query its own store name so their histories stay separate. ## `proxyConfiguration` (type: `object`): Select the proxy used for every request. Keep a rotating group enabled: GDELT rate-limits by IP, and every GDELT request and retry opens a fresh session to draw a different exit address, which is what lets multi-window runs through. ## Actor input object example ```json { "queries": [ "tesla recall" ], "presetFeeds": [], "sources": [ "google-news" ], "maxResults": 100, "resolveGoogleUrls": false, "includeFullText": false, "skipSeenArticles": false, "seenArticlesStoreName": "news-seen-articles", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } } ``` # Actor output Schema ## `articles` (type: `string`): Every article the run collected: one row per article with all 19 fields, deduplicated across Google News, GDELT and RSS. ## `headlines` (type: `string`): Headline, publisher, domain, publication date, summary and link - the columns most runs need. ## `articleText` (type: `string`): Article bodies with their character count. Populated when Include full text is on; publishers that refuse automated fetches leave the body empty and keep their summary. ## `gdeltFields` (type: `string`): Language and source country, which only GDELT rows carry. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "tesla recall" ], "presetFeeds": [], "sources": [ "google-news" ], "maxResults": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }; // Run the Actor and wait for it to finish const run = await client.actor("automly/google-news-scraper-api").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["tesla recall"], "presetFeeds": [], "sources": ["google-news"], "maxResults": 100, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US", }, } # Run the Actor and wait for it to finish run = client.actor("automly/google-news-scraper-api").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "tesla recall" ], "presetFeeds": [], "sources": [ "google-news" ], "maxResults": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }' | apify call automly/google-news-scraper-api --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,automly/google-news-scraper-api" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/ZOH1cdmLoGXVZuSse/builds/gl6Rh3vtLcGFm7vZ1/openapi.json