# Google News Scraper (`mangudai/google-news-scraper`) Actor Scrape Google News by keyword, topic, or top headlines. Get headline, source, publish time, article link, and a snippet as clean structured rows. Built on the public Google News RSS feed: no login, no API key, no captcha. Runs on Apify Proxy. - **URL**: https://apify.com/mangudai/google-news-scraper.md - **Developed by:** [MangudΓ€i](https://apify.com/mangudai) (community) - **Categories:** News, Developer tools, Open source - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.20 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google News Scraper Pull live Google News results by keyword, topic, or top headlines and get them back as clean, structured rows. Built on the public Google News RSS feed, so there is no login, no API key, and no captcha. Route it through Apify Proxy and run it on a schedule, via API, or from your own code. ### What it does Give it a search term and it returns the matching news articles: headline, source, publish time, article link, and a short snippet. Skip the search term and pick a topic (technology, business, sports, and the rest) to get that section's headlines, or leave both empty for the top headlines of the day. ### Input - `query`: keyword or phrase to search. Supports Google News operators like `when:7d`, `site:reuters.com`, and `OR`. - `additionalQueries`: extra search terms to fetch in the same run, one feed each. - `topic`: section headlines (World, Nation, Business, Technology, Entertainment, Sports, Science, Health). Used only when no query is set. - `language`: interface language, for example `en-US`, `fr`, `de`. - `country`: edition country, for example `US`, `GB`, `IN`. - `maxItemsPerQuery`: cap per feed. Google News RSS returns up to about 100 items per feed. `0` means no cap. - `proxyConfiguration`: Apify Proxy settings. Default run: searches `artificial intelligence` and returns up to 50 recent articles. ### Output One row per article: - `title`: headline with the trailing source name removed - `fullTitle`: original headline as Google returns it - `source`: publisher name - `sourceUrl`: publisher homepage - `link`: Google News article link - `guid`: stable article id - `publishedAt`: ISO 8601 timestamp - `publishedRaw`: original RFC 822 date string - `snippet`: short text description - `query`, `topic`, `feedTitle`: what produced the row ### Notes Google News RSS caps each feed at roughly 100 items. To gather more, split your work across several queries (use `additionalQueries`) or narrow with date operators such as `when:1d`. The scraper retries failed fetches and rotates the proxy IP between attempts. # Actor input Schema ## `query` (type: `string`): Keyword or phrase to search Google News. Supports Google News operators such as when:7d, site:, and OR. Leave empty to pull topic headlines or top headlines instead. ## `additionalQueries` (type: `array`): Extra search terms to fetch in the same run, one feed each. Adds to the main query. ## `topic` (type: `string`): Google News section headlines. Used only when no search query is provided. ## `language` (type: `string`): Interface language code, for example en-US, en-GB, fr, de, es. ## `country` (type: `string`): Edition country code, for example US, GB, CA, AU, IN. ## `maxItemsPerQuery` (type: `integer`): Cap on articles saved per feed. Google News RSS returns up to about 100 items per feed. Set 0 for no cap. ## `proxyConfiguration` (type: `object`): Route requests through Apify Proxy. Recommended for scale and reliability. ## Actor input object example ```json { "query": "artificial intelligence", "additionalQueries": [], "topic": "", "language": "en-US", "country": "US", "maxItemsPerQuery": 50, "proxyConfiguration": { "useApifyProxy": true } } ```` # Actor output Schema ## `articles` (type: `string`): One item per news article parsed from the Google News RSS feed. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "artificial intelligence", "additionalQueries": [], "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("mangudai/google-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "artificial intelligence", "additionalQueries": [], "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("mangudai/google-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "artificial intelligence", "additionalQueries": [], "proxyConfiguration": { "useApifyProxy": true } }' | apify call mangudai/google-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=mangudai/google-news-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Google News Scraper", "description": "Scrape Google News by keyword, topic, or top headlines. Get headline, source, publish time, article link, and a snippet as clean structured rows. Built on the public Google News RSS feed: no login, no API key, no captcha. Runs on Apify Proxy.", "version": "0.0", "x-build-id": "iMola4ZwziQnU3xuY" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/mangudai~google-news-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-mangudai-google-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/mangudai~google-news-scraper/runs": { "post": { "operationId": "runs-sync-mangudai-google-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/mangudai~google-news-scraper/run-sync": { "post": { "operationId": "run-sync-mangudai-google-news-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Search query", "type": "string", "description": "Keyword or phrase to search Google News. Supports Google News operators such as when:7d, site:, and OR. Leave empty to pull topic headlines or top headlines instead." }, "additionalQueries": { "title": "Additional search queries", "type": "array", "description": "Extra search terms to fetch in the same run, one feed each. Adds to the main query.", "items": { "type": "string" } }, "topic": { "title": "Topic (when no query is set)", "enum": [ "", "WORLD", "NATION", "BUSINESS", "TECHNOLOGY", "ENTERTAINMENT", "SPORTS", "SCIENCE", "HEALTH" ], "type": "string", "description": "Google News section headlines. Used only when no search query is provided.", "default": "" }, "language": { "title": "Language (hl)", "type": "string", "description": "Interface language code, for example en-US, en-GB, fr, de, es.", "default": "en-US" }, "country": { "title": "Country (gl)", "type": "string", "description": "Edition country code, for example US, GB, CA, AU, IN.", "default": "US" }, "maxItemsPerQuery": { "title": "Max items per query", "minimum": 0, "type": "integer", "description": "Cap on articles saved per feed. Google News RSS returns up to about 100 items per feed. Set 0 for no cap.", "default": 50 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Route requests through Apify Proxy. Recommended for scale and reliability.", "default": { "useApifyProxy": true } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```