# News Aggregator — Multi-Source RSS Monitor (`bercikgroup/news-aggregator-rss-monitor`) Actor

Monitor news across Bing, Reddit, publisher feeds and any custom RSS/Atom URL. Cross-source deduplication and keyword filtering in one dataset.

- **URL**: https://apify.com/bercikgroup/news-aggregator-rss-monitor.md
- **Developed by:** [BERCIK Group](https://apify.com/bercikgroup) (community)
- **Categories:** News, Business, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

Monitor news across **many sources at once** — Bing News, Reddit, major publisher feeds and **any RSS or Atom URL you supply** — and get back one clean, **deduplicated** dataset instead of five overlapping ones.

### What does News Aggregator do?

Media monitoring usually means watching a dozen feeds that all carry the same story. This Actor fetches every source you enable, normalises RSS 2.0 and Atom into **one schema**, and then collapses duplicates so each story appears **once**, with a record of everywhere else it turned up.

It also **unwraps aggregator redirect links**. Search feeds wrap every article in a click-tracking URL, which means the "publisher" reads as the aggregator rather than the actual outlet. This Actor resolves those back to the real article URL — so you get `forbes.com`, not `bing.com`, and the same story from a search feed correctly matches the one from a publisher feed.

No API key, no login, no browser.

### Why use News Aggregator?

- **Brand and reputation monitoring** — track every mention of your company across many outlets in one feed.
- **Competitor tracking** — watch rival launches, funding and executive moves.
- **PR measurement** — see which outlets picked up a story, and how widely, via the duplicate count.
- **Content curation** — power a newsletter or internal digest from a curated source list.
- **Market and trading research** — monitor sentiment-moving stories on specific tickers or sectors.
- **Feed consolidation** — replace a dozen RSS subscriptions with one scheduled dataset that feeds Slack, Sheets or your warehouse.

### How to use News Aggregator

1. Click **Try for free**.
2. Add **search queries** — e.g. `nvidia`, `defense contract`.
3. Pick your **search sources** (Bing News, Reddit) and any **publisher feeds** you want in full.
4. Add **custom RSS/Atom URLs** for anything else you follow — this is the most flexible option.
5. Narrow with **Must contain** / **Must not contain** keywords.
6. Click **Start**, then download as **JSON, CSV, Excel, HTML or XML**.

To run it as a monitor, schedule it hourly or daily and deduplicate downstream on `canonicalUrl`.

### Input

```json
{
  "queries": ["nvidia", "defense contract"],
  "searchSources": ["bing", "reddit"],
  "publisherFeeds": ["bbc", "techcrunch", "cnbc_business"],
  "customFeedUrls": ["https://hnrss.org/frontpage"],
  "keywords": ["chip"],
  "excludeKeywords": ["rumor"],
  "deduplicate": true,
  "maxResults": 500
}
```

| Field | Type | Description |
| --- | --- | --- |
| `queries` | array | Search terms, run against each enabled search source |
| `searchSources` | array | `bing`, `reddit`, `google_news` |
| `publisherFeeds` | array | Built-in feeds: BBC, NYT Technology, TechCrunch, Ars Technica, CNBC Business, Hacker News |
| `customFeedUrls` | array | Any RSS or Atom feed URL |
| `keywords` / `excludeKeywords` | array | Include / exclude filters on title and description |
| `publishedAfter` | string | Only articles on or after this date |
| `language`, `country` | string | Locale for query-driven sources |
| `deduplicate` | boolean | Collapse the same story across sources |
| `maxResults` | integer | Hard cap on records returned |

### Output

```json
{
  "title": "Nvidia stock gains but AMD is coming for more of the AI chip market",
  "url": "https://www.msn.com/en-us/money/other/nvidia-stock-gains/ar-AA29BDDw",
  "canonicalUrl": "https://msn.com/en-us/money/other/nvidia-stock-gains/ar-AA29BDDw",
  "publisher": "msn.com",
  "sourceLabel": "Bing News",
  "matchedQuery": "nvidia",
  "publishedAt": "2026-08-07T09:14:00.000Z",
  "description": "Revenue rose and margins held as data-centre demand continued.",
  "author": null,
  "imageUrl": null,
  "duplicateCount": 2,
  "alsoFoundIn": ["Bing News", "CNBC Business"],
  "isRedirectLink": false,
  "originalFeedUrl": "http://www.bing.com/news/apiclick.aspx?...&url=https%3a%2f%2fwww.msn.com%2f..."
}
```

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

#### Data fields

| Field | Description |
| --- | --- |
| `title`, `description`, `author` | Article content from the feed |
| `url` | Real article URL, with aggregator redirects unwrapped |
| `canonicalUrl` | Normalised URL — use this to deduplicate across scheduled runs |
| `publisher` | Host of the real article URL |
| `sourceLabel`, `source`, `matchedQuery`, `feedUrl` | Which source surfaced it and why |
| `publishedAt` | Publication time, normalised to ISO 8601 |
| `imageUrl` | Thumbnail where the feed provides one |
| `duplicateCount`, `alsoFoundIn` | How many sources carried this story, and which |
| `isRedirectLink` | True when the link could not be unwrapped to a direct URL |
| `originalFeedUrl` | The wrapped URL, when unwrapping changed it |

### How much does it cost to monitor news?

Priced **per article returned**, and you're charged only for what survives deduplication and filtering — not for the raw fetch. **Maximum results** caps your spend before the run starts.

Deduplication typically removes a meaningful share of a multi-source run, so you pay for stories rather than copies.

### Tips

- **Custom feeds are the strongest option.** Any publisher's own RSS URL works, is unambiguous to use, and gives direct article links with no redirect wrapper.
- **Publisher feeds ignore your queries** — they return the whole feed. Use **Must contain** to narrow them.
- **`imageUrl` is sparse.** Many feeds omit images entirely; roughly a quarter of items carry one.
- **Reddit rate-limits aggressively.** The Actor backs off and retries automatically, but heavy Reddit use will slow a run.
- **Deduplication is order-sensitive by design.** Publisher and custom feeds are fetched before search sources, so the surviving record keeps the direct publisher link.
- **For monitoring, schedule frequently and dedupe on `canonicalUrl`** across runs — within a run it's handled for you.

### A note on Google News

Google News is available as a source but is **off by default**. Google's feed carries an explicit notice that it is provided "solely for the purpose of rendering Google News results within a personal feed reader for personal, non-commercial use."

If you enable it, that restriction is yours to evaluate against your use case. Two practical limitations also apply: Google wraps every link in an opaque redirect that **cannot** be unwrapped without an extra request per article, so those rows keep `isRedirectLink: true` and report Google as the publisher, and they will not deduplicate against the same story from a publisher feed.

For most uses, custom publisher feeds give better data with none of this ambiguity.

### FAQ

**Is scraping RSS feeds legal?**
RSS and Atom feeds are published specifically to be read programmatically — that is their entire purpose. This Actor reads them at a polite rate and stores only the metadata the feed itself publishes. Individual feeds may still attach their own terms, as Google's does; check any source you rely on commercially. This is general information rather than legal advice.

**Does it fetch the full article text?**
No. It returns what the feed provides — usually a headline and a summary. Full text would mean fetching every article page, which raises both cost and copyright questions.

**A feed returned nothing — why?**
Some publishers block datacenter traffic outright (the Guardian and AP both do). The run log names any feed that failed and the run continues.

**How does deduplication decide two stories are the same?**
By normalised URL first, then by a normalised headline with the trailing " - Publisher" suffix removed. Same URL or same headline collapses into one record.

**Can you add more built-in publishers?**
Yes — open an issue with the feed URL. Any feed can already be used today via **Custom RSS / Atom feed URLs**.

### Support

Found a bug or want another source built in? Open an issue on the **Issues** tab. Custom data pipelines and bespoke scraping work are available on request.

# Actor input Schema

## `queries` (type: `array`):

Terms to search for. Used by the search sources below — each query is run against each enabled source.

## `searchSources` (type: `array`):

Query-driven sources. Google News is off by default: its feed states it is provided for personal, non-commercial use only.

## `publisherFeeds` (type: `array`):

Well-known publisher feeds to pull in full. These are not filtered by your queries — use Keywords below to narrow them.

## `customFeedUrls` (type: `array`):

Any RSS or Atom feed URL. This is the most flexible option — point it at any publisher, blog or alert feed you already follow.

## `keywords` (type: `array`):

Keep only articles whose title or description contains at least one of these terms.

## `excludeKeywords` (type: `array`):

Drop articles whose title or description contains any of these terms.

## `publishedAfter` (type: `string`):

Only keep articles published on or after this date. Articles with no date are kept.

## `language` (type: `string`):

Language code used by query-driven sources, e.g. en-US, de-DE.

## `country` (type: `string`):

Country code used by query-driven sources, e.g. US, GB, DE.

## `deduplicate` (type: `boolean`):

Collapse the same story appearing in multiple sources into one record, tracking where else it was found.

## `maxResults` (type: `integer`):

Hard cap on articles returned. You are charged per record.

## Actor input object example

```json
{
  "queries": [
    "nvidia"
  ],
  "searchSources": [
    "bing"
  ],
  "publisherFeeds": [],
  "language": "en-US",
  "country": "US",
  "deduplicate": true,
  "maxResults": 500
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "nvidia"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("bercikgroup/news-aggregator-rss-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["nvidia"] }

# Run the Actor and wait for it to finish
run = client.actor("bercikgroup/news-aggregator-rss-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "nvidia"
  ]
}' |
apify call bercikgroup/news-aggregator-rss-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bercikgroup/news-aggregator-rss-monitor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wRMAtaRw8vkeBLC6V/builds/7dMgM33sDtsR6JidQ/openapi.json