News Scraper by Topic + AI Briefing

Track any topic and pull recent news articles from Google News and any RSS/Atom feed — title, source, date, link, and snippet — with an optional AI briefing that summarizes everything into a few bullet points. Great for monitoring a subject, feeding fresh headlines to an LLM, or building a daily digest.

What it does

• Topic tracking — give it terms like artificial intelligence or Tesla earnings; it fetches the latest matching articles from Google News.
• Any RSS/Atom feed — add your own feed URLs to pull alongside (or instead of) topics.
• Clean, structured output — each article as { title, source, published, link, snippet, query }, de-duplicated across sources.
• Optional AI briefing — one concise summary of everything collected, as bullets, a concise paragraph, a detailed write-up, or a grouped outline.
• Reliable — built on RSS, which is designed to be fetched by machines, so it just works.

Input

Field	Type	Description
queries	array	Topics / search terms to track via Google News.
rssUrls	array	Optional specific RSS or Atom feed URLs.
language	string	Two-letter language code (default en).
country	string	Two-letter country code (default US).
maxItemsPerSource	integer	Articles to keep per topic/feed (default 25).
includeAiBriefing	boolean	Add a single AI briefing (premium). Default false.
summaryStyle	concise | bullets | detailed | chapters	Shape of the briefing. Default bullets.

At least one of queries or rssUrls is required.

Example input

{
  "queries": ["artificial intelligence", "renewable energy"],
  "language": "en",
  "country": "US",
  "maxItemsPerSource": 20,
  "includeAiBriefing": false
}

Output

Each article becomes one dataset item:

{
  "title": "New AI model sets benchmark record",
  "link": "https://news.example.com/...",
  "source": "Example News",
  "published": "Fri, 08 Aug 2026 14:03:00 GMT",
  "publishedIso": "2026-08-08T14:03:00Z",
  "snippet": "Researchers announced ...",
  "query": "artificial intelligence"
}

When includeAiBriefing is on, a final item is added:

{ "type": "briefing", "articleCount": 40, "queries": ["artificial intelligence"], "summary": "- ..." }

Notes

• Only public data is accessed (public Google News results and the feeds you provide).
• Results reflect Google News' own coverage and ranking for your language/country.

Local development

python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
apify run

The fetch core (src/news.py) has no Apify dependency and can be run directly for quick testing.