Google News Scraper

Google News Scraper

The Google News Scraper is a real-time Google News scraping tool that collects headlines, sources, timestamps, snippets, links, and thumbnails from Google News RSS results — fast and at scale. It solves the hassle of manual monitoring by automating a Google News crawler workflow for marketers, developers, analysts, and researchers. As a Python Google News scraper and Google News API alternative, it supports multilingual, multi-country searches and delivers clean, structured data you can act on — perfect for dashboards, SEO research, media monitoring, and trend analysis at scale. 🚀

What data / output can you get?

Below are the exact fields this Google News data extractor writes to the dataset (one item per result). Example values illustrate typical outputs.

Data type	Description	Example value
position	Result index within the run	1
title	Article headline	“Tesla shares rise after delivery beat”
link	Resolved article URL	https://www.example.com/business/tesla-earnings
domain	Source domain (derived from source name or link)	example.com
source	Publisher/source name (parsed from RSS title)	Bloomberg
date	Human-friendly published time (relative)	“2 hours ago”
date_utc	ISO-8601 datetime (UTC)	“2026-04-02T10:15:00+00:00”
snippet	Cleaned snippet/preview from RSS description	“Tesla reported deliveries that exceeded expectations…”
thumbnail	Base64 data URL of article thumbnail (if detected)	“data:image/jpeg;base64,/9j/4AAQSkZJRgABAQ…”
block_position	Position within the processed block (mirrors position)	1

Notes:

• Thumbnails are extracted from Open Graph/Twitter tags or article images and encoded as base64. If none is suitable, thumbnail may be an empty string.
• Export your dataset in JSON, CSV, or Excel via Apify’s built-in exports.

Key features

• ⚡ Real-time RSS feed extraction
  A fast, automated Google News RSS feed scraper that pulls fresh results for your query and time window, ideal to scrape Google News headlines continuously.

• 🌍 Region & language targeting
  Control country and language with gl (Google Country), hl (UI Language), lr (Language Results), and cr (Country Results) for localized coverage.

• ⏱️ Time filters including custom ranges
  Filter by last_hour, last_day, last_week, last_month, last_year, or set a custom window with time_period_min/time_period_max.

• 🖼️ Image enrichment with base64 thumbnails
  Follows the Google redirect to the article page, finds Open Graph/Twitter images or hero images, and returns a thumbnail as a base64 data URL.

• 🛡️ Smart proxy fallback (none → datacenter → residential)
  Starts without a proxy and automatically escalates to datacenter then residential proxies on blocks or failures for higher reliability.

• 🔁 Resilience with retries & rate limiting
  Handles transient errors with automatic retries and modest backoff while streaming results to the dataset in real time.

• 💻 Developer-friendly Python actor
  Built with the Apify Python SDK (aiohttp, BeautifulSoup, lxml). Perfect for automated Google News scraper pipelines and API-driven consumption.

• 📊 Clean, structured output
  Consistent JSON fields ready for analytics, dashboards, and research — a practical Google News API alternative for production workflows.

How to use Google News Scraper - step by step

1. Sign in to Apify
   Create a free Apify account or log in to your dashboard.

2. Open Google News Scraper
   Find and open the actor from your Apify console.

3. Add your input

   • Set query (e.g., “Tesla”) and maxItems (100–5000).
   • Optionally set gl (Google Country), hl (UI Language), lr (Language Results), cr (Country Results).
   • Choose a time_period (e.g., last_day) or use custom with time_period_min/time_period_max (MM/DD/YYYY).
   • Adjust nfpr (No Autocorrect) and filter (Similar/Omitted Results) if needed.
   • Configure proxyConfiguration or leave default (no proxy; automatic fallback logic applies during execution).

4. Start the run
   Click Start. The automated Google News scraper fetches results asynchronously and saves them to the dataset as they’re processed.

5. Monitor progress
   View logs for proxy fallback actions, retries, and how many items have been saved in real time.

6. Download your data
   Export the dataset in JSON, CSV, or Excel. Use the Apify Dataset API for programmatic access.

Pro Tip: Schedule runs to keep a real-time Google News scraper feed flowing into your BI tools or data lake.

Use cases

Use case name	Description
Media monitoring & PR tracking	Monitor breaking stories and brand mentions across regions and languages with a Google News crawler that runs on a schedule.
SEO & content strategy	Scrape Google News headlines to discover trending topics and fill content gaps faster.
Market & financial analysis	Track sector-specific news for earnings, guidance, and regulatory updates with consistent JSON output.
Competitive intelligence	Follow competitor product launches, partnerships, and press coverage across markets.
Academic & policy research	Collect reproducible, timestamped datasets for longitudinal studies and topic analysis.
Automated data pipelines	Pipe structured results into warehouses or apps via the Dataset API — an effective Google News API alternative.

Why choose Google News Scraper?

This automated Google News scraper is engineered for precision, resilience, and scale.

• ✅ Accurate, structured results ready for analysis
• 🌐 Multilingual and multi-country support via gl, hl, lr, cr
• 📈 Scales from hundreds to thousands of items per run (maxItems up to 5000)
• 🔧 Developer access with Apify’s API for automated ingestion
• 🛡️ Ethical, public-source collection via RSS + article enrichers
• 💸 Try before you buy — 120 trial minutes are available
• 🧩 More reliable than ad‑hoc extensions or manual copy-paste

Bottom line: a production-ready Google News scraping tool built for real workflows, not demos.

Is it legal / ethical to use Google News Scraper?

Yes — when done responsibly. This actor extracts data from public Google News RSS feeds and follows article links to retrieve metadata like images for thumbnails.

Guidelines:

• Respect Google’s and publishers’ terms of service.
• Use reasonable schedules and avoid abusive request patterns.
• Do not republish full content without permission; use data for compliant purposes.
• Follow applicable data protection laws (e.g., GDPR, CCPA).
• Consult your legal team for edge cases or redistribution questions.

Input parameters & output format

Example input JSON

{
  "maxItems": 300,
  "query": "Tesla",
  "gl": "United States",
  "hl": "English",
  "lr": "English",
  "cr": "United States",
  "time_period": "last_day",
  "time_period_min": "03/30/2026",
  "time_period_max": "04/02/2026",
  "nfpr": 1,
  "filter": 1,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Parameters

• maxItems (integer)
  Description: Maximum number of search results to retrieve. Min 100, Max 5000.
  Default: 100
  Required: Yes

• query (string)
  Description: The search term to use.
  Default: “Elon Musk”
  Required: Yes

• gl (string)
  Description: The Google country to use for the query (enum list of countries).
  Default: None
  Required: No

• hl (string)
  Description: The Google UI language to return results (enum list of languages).
  Default: None
  Required: No

• lr (string)
  Description: Limit the results to a specific language (enum).
  Default: None
  Required: No

• cr (string)
  Description: Limit the results to a specific country (enum).
  Default: None
  Required: No

• time_period (string)
  Description: Time period for results. One of: last_hour, last_day, last_week, last_month, last_year, custom.
  Default: None
  Required: No

• time_period_min (string)
  Description: Minimum date for custom time period (MM/DD/YYYY).
  Default: None
  Required: No

• time_period_max (string)
  Description: Maximum date for custom time period (MM/DD/YYYY).
  Default: None
  Required: No

• nfpr (integer)
  Description: Exclude results from auto-corrected queries (0 or 1).
  Default: 0
  Required: No

• filter (integer)
  Description: Enable/disable Similar Results and Omitted Results filters (0 or 1).
  Default: 1
  Required: No

• proxyConfiguration (object)
  Description: Configure proxy settings. The actor will start with no proxy, then fallback to datacenter, then residential proxies if needed.
  Default: { "useApifyProxy": false } (prefill)
  Required: No

Example output JSON

{
  "position": 1,
  "title": "Tesla shares rise after delivery beat",
  "link": "https://www.example.com/business/tesla-earnings",
  "domain": "example.com",
  "source": "Bloomberg",
  "date": "2 hours ago",
  "date_utc": "2026-04-02T10:15:00+00:00",
  "snippet": "Tesla reported deliveries that exceeded expectations for the quarter, sending shares higher in early trading...",
  "thumbnail": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQ...",
  "block_position": 1"
}

Notes:

• thumbnail may be an empty string if no suitable image is detected or fetched.
• date is a human-readable relative string; date_utc is ISO-8601 UTC.

FAQ

Is the Google News Scraper a Google News API alternative?

Yes. It delivers clean JSON from Google News RSS plus enriched fields like base64 thumbnails, making it a practical Google News API alternative for analytics and automation.

Can I scrape Google News by keyword?

Yes. Set the query parameter (e.g., “Apple”, “interest rates”) to scrape Google News search results for your topic.

Does it support multiple languages and countries?

Yes. Use hl for UI language, gl for Google country, lr to limit the language of results, and cr to limit by country.

How many results can I collect per run?

You can set maxItems between 100 and 5000. The scraper saves items continuously to the dataset as they’re processed.

Does it fetch article images?

Yes. The actor follows article links and extracts Open Graph/Twitter images or page images, returning a base64-encoded thumbnail when available.

Do I need to log in?

No. This automated Google News scraper works with public RSS endpoints and does not require login or cookies.

Is there a trial?

Yes. The listing provides 120 trial minutes so you can evaluate the scraper before subscribing.

How can I integrate the results into my app?

Consume the Apify Dataset via API or export as JSON/CSV/Excel. This makes it easy to plug into BI tools, data lakes, or Python pipelines.

Closing CTA / Final thoughts

Google News Scraper is built to automate real-time, structured collection of news headlines and metadata from Google News. It combines speed, localization controls, and robust proxy fallback with clean JSON output.

Whether you’re a marketer tracking trends, a researcher building a corpus, or a developer wiring a data pipeline, this Google News scraping tool delivers consistent, analysis-ready results. Use the Apify Dataset API or your favorite Python workflows to operationalize the data.

Start extracting smarter, faster news insights today — and keep your monitoring always-on with a reliable, automated Google News scraper. 📰✨