# Google News AI Scraper (`confidential_gnat/google-news-ai-scraper`) Actor Search Google News by keyword, optionally extract full article text and AI-generated summaries via OpenRouter, and never re-scrape the same article twice across runs via built-in cross-run deduplication. - **URL**: https://apify.com/confidential\_gnat/google-news-ai-scraper.md - **Developed by:** [ActorFlow](https://apify.com/confidential_gnat) (community) - **Categories:** - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.05 / ai data enrichment This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google News AI Scraper Search Google News by keyword and get structured article data — title, source, publish date, and link — with optional full-article-text extraction and AI-generated summaries or paraphrases. Runs against **news.google.com**, the public Google News search index. **Target website:** [news.google.com](https://news.google.com) ### ✨ Features - **News article extraction** — title, publisher, publish date, link, and snippet for each search result - **Full-text extraction** — optionally visit each article and extract the cleaned, full article body - **AI enrichment** — optionally summarize, paraphrase, extract keywords, run sentiment analysis, or apply a custom instruction to each article via an OpenRouter model - **Cross-run deduplication caching** — set a project name and the actor automatically skips articles it already returned in a previous run under the same Apify account - **Proxy support** — optional Apify Proxy configuration - **Query filtering** — language, country, date range (or absolute date window), site filter, and excluded words ### 🔧 Input Configuration | Field | Type | Required | Default | Description | | ---------------------- | ---------------- | -------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | `projectName` | String | — | — | Set to persist deduplication across runs; articles already scraped for this project are skipped. | | `queries` | Array of strings | ✅ | `["artificial intelligence"]` | Search queries to run on Google News. | | `maxResultsPerQuery` | Integer | — | `3` | Max articles returned per query, after deduplication. | | `language` | String | — | `en` | 2-letter BCP-47 language code. | | `country` | String | — | `US` | ISO 3166-1 alpha-2 country code. | | `dateRange` | Select | — | `any` | `1h` / `1d` / `7d` / `1m` / `1y` / `any`. Ignored if `dateFrom`/`dateTo` is set. | | `dateFrom` | String | — | — | Only articles on/after this date (`YYYY-MM-DD`). | | `dateTo` | String | — | — | Only articles on/before this date (`YYYY-MM-DD`). | | `siteFilter` | String | — | — | Restrict results to one domain, e.g. `reuters.com`. | | `excludeWords` | Array of strings | — | — | Words that must not appear in results. | | `includeImages` | Boolean | — | `false` | Fetch the article's main image during extraction. Visits the article page with a browser. Not currently included in the output. | | `extractFullText` | Boolean | — | `false` | Extract the full, cleaned article text via a browser visit. Automatically enabled when `aiEnabled` is on. | | `aiEnabled` | Boolean | — | `false` | Enable AI enrichment via OpenRouter. | | `aiApiKey` | String (secret) | — | — | Your OpenRouter API key. Optional — if empty, this Actor's own key is used and a per-article platform fee applies. | | `aiModel` | Select | — | `openai/gpt-4o-mini` | OpenRouter model used for AI enrichment. | | `aiFeatures` | Array (select) | — | `["summarize"]` | `summarize` / `paraphrase` / `keywords` / `sentiment` / `custom`. | | `aiCustomInstructions` | String | — | — | Extra instructions used when `custom` is selected. | | `proxyConfiguration` | Proxy object | — | `{ "useApifyProxy": false }` | Optional Apify Proxy configuration. | ### 📦 Output One dataset view (`Overview`) showing title, source, publish date, query, AI output, and link. **Sample output:** ```json { "query": "artificial intelligence", "title": "US efforts to secure AI supply chains doomed to fail: Chinese academic - South China Morning Post", "link": "https://www.scmp.com/news/china/diplomacy/article/3364973/us-efforts-secure-ai-supply-chains-are-doomed-fail-says-chinese-academic", "source": "South China Morning Post", "publishedAt": "2026-08-23T11:00:09.000Z", "description": "US efforts to secure AI supply chains doomed to fail: Chinese academic South China Morning Post", "fullText": "The latest US artificial intelligence (AI) and supply chain initiatives are doomed to fail, according to a prominent political scientist who also argued that efforts to restrict US companies’ access to low-cost Chinese models would backfire... (line truncated to 2000 chars)", "ai": { "summary": "A prominent Chinese political scientist has expressed concerns that the US' AI and supply chain initiatives are doomed to fail due to efforts to restrict US companies' access to low-cost Chinese AI models. He suggested that the US and China collaborate on driving global economic development to stimulate external demand for long-term growth.", "paraphrase": "A prominent Chinese political scientist, Zheng Yongnian, has expressed concerns that the United States' latest artificial intelligence (AI) and supply chain initiatives are doomed to fail... (line truncated to 2000 chars)", "keywords": [ "US AI initiatives", "Supply chain rivalry", "China-US economic relations", "Artificial intelligence", "Global economic development", "Commercial reality", "Market competition", "AI-driven growth model", "Systemic risks", "Global economy" ], "sentiment": "negative", "custom": null }, "scrapedAt": "2026-08-30T08:03:58.202Z" } ``` ### 💡 Uses of This Data - Ongoing brand or competitor mention monitoring - Building a topic-specific news feed or newsletter - Feeding AI-summarized news into an internal dashboard - Market and sentiment research over recent coverage - Tracking coverage of a specific publisher or domain ### 🚀 How to Use 1. [Sign up](https://apify.com/sign-up) for a free Apify account — includes **$5 monthly credit**. 2. Open the actor page and click **Try for free**. 3. Fill in **Search queries** (required), and optionally a **Project name** to enable caching. 4. Click **Start** and wait for the run to complete. 5. Download results from the **Output** tab in JSON, CSV, or Excel format. You can also run this actor via the [Apify API](https://docs.apify.com/api/v2) or integrate it directly into your workflows using [Zapier](https://zapier.com/apps/apify), [Make](https://www.make.com/), or [n8n](https://n8n.io/). ### ⚠️ Limitations & Known Issues - **Maximum ~100 results per query** — Google News RSS search feeds do not expose pagination, so the actor cannot retrieve more than the feed's limit no matter how high `maxResultsPerQuery` is set. - **Paywalled articles** — Full-text extraction may return an empty or partial body for paywalled publishers; the article is still returned with its search-result snippet. - **Rate limiting** — Very high query volumes may be rate-limited by Google News; space out large runs if you see missing results. ### 📝 Notes - **API integration** — This actor can be called as an API from any automation platform (Zapier, Make, n8n, custom scripts). - **Caching is per project name** — runs with different project names (or no project name) do not share deduplication state. - **AI enrichment pricing** — Bring your own OpenRouter API key and there's no extra fee (you're billed by OpenRouter directly). Leave the key empty and this Actor uses its own key instead, charging a small per-article platform fee. - **AI accuracy** — AI-generated summaries, paraphrases, keywords, and sentiment are based on the model's interpretation and may contain errors or hallucinations. Please fact-check any AI-enriched content before relying on it for critical decisions. ### 🔗 Other Actors | Scraper | Description | | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [🚗 Cheapest Cars & Bids Scraper](https://apify.com/confidential_gnat/cheapest-carsandbids-scraper) | Scrapes the cheapest vehicle listings from Cars & Bids, making it easy to discover the best-value auction deals. | | [🔍 Incidecoder Scraper](https://apify.com/confidential_gnat/whois-com) | Extracts cosmetic ingredient and product information from Incidecoder for research and analysis. | | [🎟️ Church Finder Scraper \[💰Free\]](https://apify.com/confidential_gnat/churchfinder-scraper) | Scrape church listings and profiles from Church Finder. Extracts church name, denomination, address, phone number, service times, ratings and reviews from city search pages or individual church URLs. | | [🔴 BidNet Direct \[Closed Links\] Bid Opportunities Scraper](https://apify.com/confidential_gnat/bidnetdirect-closed-bid-opportunities-scraper) | Scrapes closed🔴 government bid and RFP solicitations from BidNet Direct, including title, issuing organization, location, publication date, closing date, and days since closing | | [✅ BidNet Direct \[Open Links\] Bid Opportunities Scraper](https://apify.com/confidential_gnat/bidnetdirect-open-bid-opportunities-scraper) | Scrapes open✅ government bid and RFP solicitations from BidNet Direct, including title, publication date, closing date, and days remaining until closing. | ### ⚖️ Legality of this actor This actor only collects article metadata and text that is already publicly visible on Google News and the linked publisher pages — no login, paywall bypass, or private content is accessed. Scraping publicly available data is generally considered legal (see *hiQ Labs v. LinkedIn*). You are responsible for complying with the target sites' Terms of Service and applicable laws (e.g. GDPR/CCPA) if you process personal data found in scraped content. ### 💡 Use Case Teams that need an ongoing feed of news mentions for a topic, brand, or competitor — without re-processing articles they've already collected in a previous run. ### 🏭 Industry Media & Entertainment, Marketing & Advertising, Market Research, Financial Services ### 📤 Output Structured JSON dataset (exportable to CSV/Excel) with one record per article: query, title, link, source, publish date, description, full text, and AI-generated fields. ### 🌐 Domain News search and article extraction (Google News) ### 🏷️ Label `google-news`, `news-scraper`, `ai-summarization`, `openrouter`, `deduplication`, `rss` ### 💬 Support & Contact If you encounter any issues or have questions, please [open an issue](https://apify.com/confidential_gnat/google-news-ai-scraper/issues/open) You can also find more of our actors on the [Actor Flow ](https://apify.com/confidential_gnat). # Actor input Schema ## `projectName` (type: `string`): Articles already scraped under this project name in a previous run are skipped. Leave empty to disable cross-run caching. ## `queries` (type: `array`): Keywords or phrases to search on Google News. Each query is scraped separately. ## `maxResultsPerQuery` (type: `integer`): Maximum number of articles to return for each search query, after deduplication. ## `language` (type: `string`): 2-letter BCP-47 language code for Google News results (e.g. 'en', 'de', 'fr', 'es'). ## `country` (type: `string`): ISO 3166-1 alpha-2 country code for regional news (e.g. 'US', 'GB', 'DE', 'IN'). ## `dateRange` (type: `string`): Filter articles by recency. Ignored if Date from / Date to is set. ## `dateFrom` (type: `string`): Only return articles published on or after this date (YYYY-MM-DD). Overrides Date range. ## `dateTo` (type: `string`): Only return articles published on or before this date (YYYY-MM-DD). Overrides Date range. ## `siteFilter` (type: `string`): Restrict results to a specific domain (e.g. 'reuters.com', 'bbc.com'). ## `excludeWords` (type: `array`): Words that must not appear in results. ## `includeImages` (type: `boolean`): Extract the article's main image (og:image). Note: this makes the Actor visit each article page with a browser, even if Extract full text is off. ## `extractFullText` (type: `boolean`): Visit each article with a browser and extract the full, cleaned article text (not just the search-result snippet). Automatically turned on when AI enrichment is enabled, since enrichment quality depends on having the full text. ## `aiEnabled` (type: `boolean`): Run each article through an OpenRouter model to generate the AI outputs selected below. ## `aiApiKey` (type: `string`): Your OpenRouter API key (https://openrouter.ai/keys). Optional: if left empty while AI enrichment is enabled, this Actor's own key is used instead and a small per-article platform fee is charged. Never logged or stored in output. ## `aiModel` (type: `string`): OpenRouter model used for AI enrichment. ## `aiFeatures` (type: `array`): What the AI should generate for each article. ## `aiCustomInstructions` (type: `string`): Extra instructions for the AI, used when 'Custom instruction' is selected above (e.g. 'Extract any stock tickers mentioned'). ## `proxyConfiguration` (type: `object`): Google News and most publisher sites don't require a proxy. Enable one only if you see blocked requests. ## Actor input object example ```json { "queries": [ "artificial intelligence" ], "maxResultsPerQuery": 1, "language": "en", "country": "US", "dateRange": "any", "includeImages": false, "extractFullText": false, "aiEnabled": false, "aiModel": "openai/gpt-4o-mini", "aiFeatures": [ "summarize" ], "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "artificial intelligence" ], "maxResultsPerQuery": 1 }; // Run the Actor and wait for it to finish const run = await client.actor("confidential_gnat/google-news-ai-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["artificial intelligence"], "maxResultsPerQuery": 1, } # Run the Actor and wait for it to finish run = client.actor("confidential_gnat/google-news-ai-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "artificial intelligence" ], "maxResultsPerQuery": 1 }' | apify call confidential_gnat/google-news-ai-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,confidential_gnat/google-news-ai-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/R7Eyho3PuCwGEy2tl/builds/Ti8FQdgds2kjMBLsm/openapi.json