Google News Scraper Fast (Real Publisher URL Decoder + Full-Article AI/RAG Markdown)

Extract real-time Google News search results and topic feeds across 85+ country/language editions, automatically decode opaque news.google.com/rss/articles/CBMi... (AU_yqL / Fbv4je) redirect tokens into real publisher URLs, and extract clean full-article Markdown + token-bounded RAG chunks with zero residential proxy requirement.

Built as a 100% drop-in replacement for data_xplorer/google-news-scraper-fast and scrapestorm/google-news-scraper-fast-cheap-pay-per-results, designed for AI agents, Model Context Protocol (MCP) servers, LangChain / LlamaIndex RAG pipelines, PR monitoring, and quantitative sentiment engines.

---

Why Switch to This Google News Scraper?

Capability	Legacy / Abandoned Google News Scrapers	Google News Scraper Fast (This Actor)
Publisher URLs	Returns broken/opaque news.google.com/rss/articles/CBMi... links	Decodes the real publisher URL (reuters.com, techcrunch.com, etc.) via Fbv4je RPC
Full Article Text	Headline & 1-sentence snippet only	Full-Article Clean Markdown (articleMarkdown) + OpenGraph image & author
AI / RAG Readiness	Requires secondary web scraper + chunker	Built-in token-bounded ragChunks ready for vector embedding
Multi-Source Clusters	Single link per item	Extracts full relatedCoverage story cluster across competing news outlets
Proxy Cost	Forces expensive RESIDENTIAL proxies by default	Zero proxy required (direct HTTP/2 extraction)
Input Schema	Single keyword or incompatible parameters	Accepts both keyword (data_xplorer) & keywords (scrapestorm) + topics

---

Key Features

• Real Publisher URL Decoder (Fbv4je / CBMi): Automatically resolves Google News's modern AU_yqL encrypted article tokens into direct canonical publisher URLs.
• Full-Article HTML-to-Markdown Engine: Visits the decoded publisher URL and extracts JSON-LD (NewsArticle), OpenGraph (og:image, og:description, article:author), and clean Markdown body text stripped of ads, navbars, and cookie banners.
• Token-Bounded RAG Chunks (ragChunks): Splits full articles into configurable word-bounded chunks (chunkSizeWords, default 250) with attached metadata (title, source, url, publishedAt).
• 85+ Country & Language Editions (region_language): Supports every Google News COUNTRY:lang edition (US:en, GB:en, DE:de, FR:fr, ES:es, JP:ja, BR:pt-419, IN:en, etc.).
• Official Topic Feeds (topics): Scrape TOP_STORIES, WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, and HEALTH alongside custom search queries.

---

Input Example

{
  "keywords": ["Artificial Intelligence", "OpenAI"],
  "topics": ["TECHNOLOGY"],
  "timeframe": "7d",
  "region_language": "US:en",
  "maxArticles": 20,
  "decodeGoogleNewsUrls": true,
  "extractArticleContent": true,
  "chunkSizeWords": 250
}

Legacy data_xplorer/google-news-scraper-fast Input (100% Compatible)

{
  "keyword": "NVIDIA",
  "maxArticles": 10,
  "timeframe": "1d",
  "region_language": "US:en"
}

---

Output Example

{
  "keyword": "Artificial Intelligence",
  "title": "Join Discerning AI: Exploring artificial intelligence through Marquette's mission",
  "source": "Marquette Today",
  "sourceDomainUrl": "https://today.marquette.edu",
  "publishedAt": "2026-10-14T15:48:02+00:00",
  "publishedTimestamp": 1791992882,
  "url": "https://today.marquette.edu/2026/10/join-discerning-ai-exploring-artificial-intelligence-through-marquettes-mission/",
  "googleNewsUrl": "https://news.google.com/rss/articles/CBMitwFBVV95cUxPYk1i...",
  "urlDecoded": true,
  "imageUrl": "https://today.marquette.edu/wp-content/uploads/2026/10/ai-mission.jpg",
  "author": "Marquette University",
  "description": "Marquette University hosts a symposium on ethical artificial intelligence...",
  "wordCount": 412,
  "articleMarkdown": "# Join Discerning AI: Exploring artificial intelligence...\n\nMarquette University faculty and students gathered...",
  "ragChunks": [
    {
      "chunkIndex": 0,
      "wordCount": 250,
      "title": "Join Discerning AI: Exploring artificial intelligence through Marquette's mission",
      "source": "Marquette Today",
      "url": "https://today.marquette.edu/2026/10/join-discerning-ai-exploring-artificial-intelligence-through-marquettes-mission/",
      "publishedAt": "2026-10-14T15:48:02+00:00",
      "text": "# Join Discerning AI: Exploring artificial intelligence..."
    }
  ],
  "relatedCoverageCount": 3,
  "relatedCoverage": [
    {
      "title": "Join Discerning AI: Exploring artificial intelligence through Marquette's mission",
      "source": "Marquette Today",
      "googleNewsUrl": "https://news.google.com/rss/articles/..."
    }
  ],
  "scrapedAt": "2026-10-14T18:30:00+00:00"
}

---

Explore the Full Zero-Proxy AI/RAG & B2B Lead Suite (@church_of_loujess)

All 7 Actors run on 256 MB RAM with zero external proxies and support 1-click MCP integration for Claude Desktop, Cursor, and n8n:

Actor	Best For
YouTube Transcript, Chapters & AI/RAG Scraper	Android InnerTube captions, chapters, iSummaryMarkdown &
agChunks
YouTube Channel Email, Socials & B2B Leads Scraper	Creator/business emails, socials & external bio-link domain crawling
Google Trends Fast Scraper (No 429)	Interest over time, regional breakdown, rising queries & trending searches
Google News Scraper Fast (Real Decoded URLs + Markdown)	Decodes CBMi.../AU_yqL URLs via Fbv4je RPC + full article Markdown
Website Email, Phone, Socials & Tech Stack Finder	De-obfuscated emails, live DNS/MX mail provider check & 50+ tech signals
Fast Website Content & AI/RAG Markdown Crawler	Parallel sitemap.xml + BFS Markdown crawler & token-bounded
agChunks
Yahoo Finance Stock Quotes, Financials, OHLCV & AI News API	Real-time quotes, OHLCV bars, full Income/Balance/CashFlow & AI tear-sheets

• Open-Source MCP Config, LangChain Loader & n8n Templates: https://github.com/Churchofloujess/apify-zero-proxy-mcp-suite