News Aggregator — Multi-Source RSS Monitor

Monitor news across many sources at once — Bing News, Reddit, major publisher feeds and any RSS or Atom URL you supply — and get back one clean, deduplicated dataset instead of five overlapping ones.

What does News Aggregator do?

Media monitoring usually means watching a dozen feeds that all carry the same story. This Actor fetches every source you enable, normalises RSS 2.0 and Atom into one schema, and then collapses duplicates so each story appears once, with a record of everywhere else it turned up.

It also unwraps aggregator redirect links. Search feeds wrap every article in a click-tracking URL, which means the "publisher" reads as the aggregator rather than the actual outlet. This Actor resolves those back to the real article URL — so you get forbes.com, not bing.com, and the same story from a search feed correctly matches the one from a publisher feed.

No API key, no login, no browser.

Why use News Aggregator?

• Brand and reputation monitoring — track every mention of your company across many outlets in one feed.
• Competitor tracking — watch rival launches, funding and executive moves.
• PR measurement — see which outlets picked up a story, and how widely, via the duplicate count.
• Content curation — power a newsletter or internal digest from a curated source list.
• Market and trading research — monitor sentiment-moving stories on specific tickers or sectors.
• Feed consolidation — replace a dozen RSS subscriptions with one scheduled dataset that feeds Slack, Sheets or your warehouse.

How to use News Aggregator

1. Click Try for free.
2. Add search queries — e.g. nvidia, defense contract.
3. Pick your search sources (Bing News, Reddit) and any publisher feeds you want in full.
4. Add custom RSS/Atom URLs for anything else you follow — this is the most flexible option.
5. Narrow with Must contain / Must not contain keywords.
6. Click Start, then download as JSON, CSV, Excel, HTML or XML.

To run it as a monitor, schedule it hourly or daily and deduplicate downstream on canonicalUrl.

Input

{
  "queries": ["nvidia", "defense contract"],
  "searchSources": ["bing", "reddit"],
  "publisherFeeds": ["bbc", "techcrunch", "cnbc_business"],
  "customFeedUrls": ["https://hnrss.org/frontpage"],
  "keywords": ["chip"],
  "excludeKeywords": ["rumor"],
  "deduplicate": true,
  "maxResults": 500
}

Field	Type	Description
queries	array	Search terms, run against each enabled search source
searchSources	array	bing, reddit, google_news
publisherFeeds	array	Built-in feeds: BBC, NYT Technology, TechCrunch, Ars Technica, CNBC Business, Hacker News
customFeedUrls	array	Any RSS or Atom feed URL
keywords / excludeKeywords	array	Include / exclude filters on title and description
publishedAfter	string	Only articles on or after this date
language, country	string	Locale for query-driven sources
deduplicate	boolean	Collapse the same story across sources
maxResults	integer	Hard cap on records returned

Output

{
  "title": "Nvidia stock gains but AMD is coming for more of the AI chip market",
  "url": "https://www.msn.com/en-us/money/other/nvidia-stock-gains/ar-AA29BDDw",
  "canonicalUrl": "https://msn.com/en-us/money/other/nvidia-stock-gains/ar-AA29BDDw",
  "publisher": "msn.com",
  "sourceLabel": "Bing News",
  "matchedQuery": "nvidia",
  "publishedAt": "2026-08-07T09:14:00.000Z",
  "description": "Revenue rose and margins held as data-centre demand continued.",
  "author": null,
  "imageUrl": null,
  "duplicateCount": 2,
  "alsoFoundIn": ["Bing News", "CNBC Business"],
  "isRedirectLink": false,
  "originalFeedUrl": "http://www.bing.com/news/apiclick.aspx?...&url=https%3a%2f%2fwww.msn.com%2f..."
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data fields

Field	Description
title, description, author	Article content from the feed
url	Real article URL, with aggregator redirects unwrapped
canonicalUrl	Normalised URL — use this to deduplicate across scheduled runs
publisher	Host of the real article URL
sourceLabel, source, matchedQuery, feedUrl	Which source surfaced it and why
publishedAt	Publication time, normalised to ISO 8601
imageUrl	Thumbnail where the feed provides one
duplicateCount, alsoFoundIn	How many sources carried this story, and which
isRedirectLink	True when the link could not be unwrapped to a direct URL
originalFeedUrl	The wrapped URL, when unwrapping changed it

How much does it cost to monitor news?

Priced per article returned, and you're charged only for what survives deduplication and filtering — not for the raw fetch. Maximum results caps your spend before the run starts.

Deduplication typically removes a meaningful share of a multi-source run, so you pay for stories rather than copies.

Tips

• Custom feeds are the strongest option. Any publisher's own RSS URL works, is unambiguous to use, and gives direct article links with no redirect wrapper.
• Publisher feeds ignore your queries — they return the whole feed. Use Must contain to narrow them.
• imageUrl is sparse. Many feeds omit images entirely; roughly a quarter of items carry one.
• Reddit rate-limits aggressively. The Actor backs off and retries automatically, but heavy Reddit use will slow a run.
• Deduplication is order-sensitive by design. Publisher and custom feeds are fetched before search sources, so the surviving record keeps the direct publisher link.
• For monitoring, schedule frequently and dedupe on canonicalUrl across runs — within a run it's handled for you.

A note on Google News

Google News is available as a source but is off by default. Google's feed carries an explicit notice that it is provided "solely for the purpose of rendering Google News results within a personal feed reader for personal, non-commercial use."

If you enable it, that restriction is yours to evaluate against your use case. Two practical limitations also apply: Google wraps every link in an opaque redirect that cannot be unwrapped without an extra request per article, so those rows keep isRedirectLink: true and report Google as the publisher, and they will not deduplicate against the same story from a publisher feed.

For most uses, custom publisher feeds give better data with none of this ambiguity.

FAQ

Is scraping RSS feeds legal? RSS and Atom feeds are published specifically to be read programmatically — that is their entire purpose. This Actor reads them at a polite rate and stores only the metadata the feed itself publishes. Individual feeds may still attach their own terms, as Google's does; check any source you rely on commercially. This is general information rather than legal advice.

Does it fetch the full article text? No. It returns what the feed provides — usually a headline and a summary. Full text would mean fetching every article page, which raises both cost and copyright questions.

A feed returned nothing — why? Some publishers block datacenter traffic outright (the Guardian and AP both do). The run log names any feed that failed and the run continues.

How does deduplication decide two stories are the same? By normalised URL first, then by a normalised headline with the trailing " - Publisher" suffix removed. Same URL or same headline collapses into one record.

Can you add more built-in publishers? Yes — open an issue with the feed URL. Any feed can already be used today via Custom RSS / Atom feed URLs.

Support

Found a bug or want another source built in? Open an issue on the Issues tab. Custom data pipelines and bespoke scraping work are available on request.