Google News Scraper — Articles for AI & RAG

Extract Google News articles by keyword, topic, or URL with full-text extraction optimized for AI and RAG pipelines. Get headlines, sources, snippets, images, authors, and clean article text in structured JSON. Perfect for sentiment analysis, brand monitoring, competitive intelligence, and media tracking.

Features

• Multi-input mode — Search by keyword, browse by topic, or provide direct Google News URLs
• Full-text extraction — Visit article pages and extract clean text without ads, navigation, or boilerplate
• AI/RAG-ready output — Semantic field names, structured metadata, word count, and reading time estimates
• 25+ countries & 17 languages — Localized results for global news coverage
• Time filtering — Filter by last hour, day, week, month, or year
• URL decoding — Automatically resolves Google News redirect URLs to original article links
• RSS-first approach — Uses Google News RSS feeds for reliable, fast data extraction
• Article metadata — Extracts Open Graph tags, author, keywords, and description from publisher pages

Use Cases

• RAG pipelines & LLM training — Feed clean article text into vector databases and language models
• Brand monitoring & PR tracking — Track mentions across thousands of news sources in real time
• Sentiment analysis — Collect structured news data for NLP and sentiment models
• Competitive intelligence — Monitor competitors, market trends, and industry developments
• Media aggregation — Build custom news feeds combining topics, keywords, and regions
• Content curation — Discover trending stories and curate content for newsletters and blogs

Input

Parameter	Type	Default	Description
searchQueries	Array of strings	["technology"]	Keyword searches. Supports Google operators (OR, -, "exact", site:).
topics	Array of strings	[]	News topics: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH
startUrls	Array of URLs	[]	Direct Google News URLs to scrape
maxResults	Integer	50	Maximum articles per search query or topic (1–5000)
country	String	US	Country code for localized results (25+ countries)
language	String	en	Language code (17 languages supported)
timePeriod	String	1d	Time filter: 1h, 1d, 7d, 30d, 1y, all
extractFullText	Boolean	false	Extract full article text from publisher pages (slower, essential for AI/RAG)
decodeUrls	Boolean	true	Decode Google redirect URLs to original article links
proxyConfiguration	Object	Apify Proxy	Proxy settings for scraping

Output

Each article contains:

Field	Type	Description
title	String	Article headline
url	String | null	Decoded original article URL (when decodeUrls enabled)
source	String | null	Publisher name (e.g., "BBC News", "Reuters")
publisher_url	String | null	Publisher website URL (e.g., "https://www.reuters.com")
author	String | null	Article author (when extractFullText enabled)
published_at	String | null	Publication timestamp (ISO 8601)
snippet	String | null	Article summary from Google News
image_url	String | null	Article thumbnail image URL
category	String | null	News category (e.g., "TECHNOLOGY", "BUSINESS")
topic	String | null	Google News topic when browsing by topic
search_query	String | null	The search query that produced this result
full_text	String | null	Full article text (when extractFullText enabled)
word_count	Integer | null	Word count of extracted text
estimated_reading_time_min	Integer | null	Estimated reading time in minutes
article_metadata	Object | null	Open Graph and meta tags from the article page
scraped_at	String	ISO 8601 extraction timestamp
source_url	String	Google News feed URL that produced this result
publisher_url	String | null	Publisher website URL (e.g., "https://www.reuters.com")

Pricing

This actor uses Pay-Per-Event pricing:

Event	Price	Description
result	$0.003	Per article with headline, source, snippet, and metadata
full-text-result	$0.010	Per article with full text, author, word count, and reading time

Examples:

• 100 articles (headlines only) = $0.30
• 100 articles (with full text) = $1.00
• 1,000 articles (headlines only) = $3.00
• 1,000 articles (with full text) = $10.00

Tips & Limitations

• Start without extractFullText for fast headline-only results, then enable it when you need article body text
• Google News RSS feeds return up to ~100 articles per search query; use multiple queries or topics for broader coverage
• Time period filtering only applies to keyword searches, not topic pages
• For best results with AI/RAG pipelines, enable extractFullText to get clean article text
• The decodeUrls option resolves Google News redirect URLs — keep it enabled for original article links
• Use Apify Proxy with residential rotation if you encounter rate limiting

Integration Examples

API

curl -X POST "https://api.apify.com/v2/acts/USERNAME~google-news-scraper/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries": ["AI technology", "climate change"], "maxResults": 50, "extractFullText": true}'

Schedule

Set up scheduled runs via Apify Console for continuous news monitoring and brand tracking.

MCP Integration

This actor is optimized for AI agents via the Apify MCP server. Semantic field names and rich dataset schemas make it easy for LLMs to understand and use the output.

What is Google News Scraper?

Google News Scraper turns the target data into structured, reusable results on Apify. Use it when you need repeatable collection for analysts, developers, agencies, researchers, and AI-agent workflows without maintaining a custom scraper or one-off integration. Run it manually, schedule recurring jobs, call it through the Apify API, or connect it to an AI agent through the Apify MCP server.

The Actor stores results in an Apify dataset, where they can be previewed and exported as JSON, CSV, Excel, XML, or RSS. Availability and completeness depend on the source, supplied inputs, public visibility, authentication requirements, and upstream rate limits.

Use cases for Google News Scraper

• Build structured datasets for research, reporting, enrichment, or monitoring.
• Automate repetitive collection with schedules, webhooks, and API calls.
• Feed clean records into spreadsheets, databases, CRMs, BI tools, AI agents, or RAG pipelines.
• Track changes over time by running the same validated input on a schedule.
• Replace fragile manual copy-and-paste work with a reproducible Apify workflow.

How to use Google News Scraper

1. Open the Actor input page and choose a focused, valid target.
2. Set a conservative result limit for the first run.
3. Start the Actor and inspect the dataset for coverage and field availability.
4. Export the results or connect the dataset to your downstream system.
5. Scale gradually and use scheduling, pagination, or proxies when supported.

Important input options

• searchQueries — List of keyword searches to run on Google News. Supports Google search operators: use OR for multiple terms, minus sign to exclude words, quotes for exact match, site: to filter by source. U
• topics — Select Google News topic pages to browse. Use this when the user wants to browse top headlines by category, not search for a specific keyword.
• startUrls — Direct Google News URLs to scrape. Use this field when the user provides a specific Google News URL. Do NOT use this when the user describes a topic or wants to search by keyword — use searc
• maxResults — Maximum number of news articles to extract per search query or topic. Each article counts as one result for billing. Lower values complete faster.
• country — Country code for localized Google News results. Affects which regional edition of Google News is scraped.
• language — Language code for Google News results. Affects the interface language and may affect which articles are shown.
• timePeriod — Filter news articles by publication date. Only applies to keyword searches — topic pages show their own curated results.
• extractFullText — Visit each article URL and extract the full text content. Essential for AI/RAG pipelines and sentiment analysis. Increases runtime and cost per result. When disabled, returns headline, snipp

API and automation example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('muhammadafzal/google-news-scraper').call({
  // Add the same input fields you use in the Apify Console.
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Related Apify Actors

Use these dedicated tools when a neighboring data source or workflow is a better match:

• Weather MCP Server — Forecast, Historical & Air Quality
• Google Scholar Scraper — Academic Papers & Citations
• Email Validator — Syntax, MX & Risk Checks
• GitHub Repo Search — Stars, Language & Topics
• Contractor Lead Scraper — Verified Emails (HVAC, Plumbing)
• Craigslist Scraper — Extract Listings, Prices & Locations
• AutoScout24 Car Listings Scraper — Prices, Specs & Details
• Domain.com.au Property Scraper — Sale, Rent & Sold Listings
• StepStone Jobs Scraper — Job Listings & Career Data
• eBay Scraper — Product Listings, Prices & Seller Data

Frequently asked questions

How many results can I scrape with Google News Scraper?

The practical total depends on the source, input limits, pagination, available records, run timeout, and upstream restrictions. Start with a small run, verify the output, and increase the limit gradually.

Can I integrate Google News Scraper with other apps?

Yes. Use Apify integrations, webhooks, schedules, dataset exports, Make, Zapier, Google Sheets, cloud storage, or your own application.

Can I use Google News Scraper with the Apify API?

Yes. Start runs with the Apify REST API or an official Apify client, then retrieve records from the run's default dataset. Keep your API token in a secret or environment variable.

Can I use Google News Scraper through an MCP Server?

Yes. The Apify MCP server can expose the Actor to compatible AI clients and agents. Review the input and expected cost before allowing an autonomous workflow to run it at scale.

Do I need proxies?

It depends on the source and volume. Use the default configuration first. For larger or geographically sensitive jobs, select an appropriate proxy configuration only when the Actor supports it.

Is it legal to scrape this data?

Scraping rules vary by source, jurisdiction, data type, and intended use. Collect only data you are authorized to access, respect applicable terms and privacy laws, and avoid restricted or personal data misuse. This documentation is not legal advice.

Your feedback

If a field is missing, a source layout has changed, or you need a supported use case documented, open an issue on the Actor page with a reproducible input and run ID.