RSS News Aggregator

📰 RSS News Aggregator – Parse RSS & Atom Feeds into Structured News Data

The RSS News Aggregator turns any list of RSS or Atom feed URLs into clean, normalised JSON news data. Give it a set of feeds — central bank press releases, financial newswires, competitor blogs, security advisories, industry publications — and it fetches each one, parses the XML, strips the HTML out of every summary, converts publication dates to ISO 8601, and returns one dataset item per source containing the full list of articles.

Feed formats are a mess in practice. Some publishers emit RSS 2.0, others Atom; some put the body in summary, others in description; dates arrive in half a dozen formats and article bodies are riddled with markup. This RSS aggregator absorbs those differences and gives you a single consistent article shape across every feed you monitor, so the code consuming it never has to care which standard a publisher chose. If you would rather handle that yourself, a raw mode returns the untouched feed structure instead.

---

📊 What Data Can You Extract with This RSS Feed Scraper?

Each dataset item represents one feed, with the articles nested inside it.

Category	Fields	What you get
🌐 Feed identity	source	The domain the feed was fetched from, e.g. federalreserve.gov
📄 Article collection	feeds	The array of parsed articles from that feed, or the raw channel structure in raw mode
📝 Article content	feeds[].title, feeds[].description	Headline plus the summary text with HTML markup stripped out
🔗 Article links & identity	feeds[].link, feeds[].guid	The canonical article URL and the feed's own unique identifier for the entry
🕒 Publication timing	feeds[].pub_date	Publication or update time normalised to ISO 8601 in UTC
🏷️ Attribution & classification	feeds[].author, feeds[].category, feeds[].source	Byline, the entry's first category tag, and the originating domain
📎 Attached media	feeds[].enclosure	Enclosure URL, MIME type and length when the entry carries audio, video or an image

The field that quietly does the most work is pub_date. Because it is converted to ISO 8601 UTC regardless of what the publisher emitted, you can sort and window articles across a dozen feeds from different countries without writing a single date-parsing branch.

---

🌟 Key Features of the RSS News Aggregator

Feature	Description
🔀 RSS and Atom support	Both standards are parsed into the same normalised article shape, so mixed feed lists just work
🧼 HTML-stripped summaries	Article descriptions are cleaned of markup, leaving readable plain text ready for analysis
🕐 ISO 8601 timestamps	Publication and update dates are converted to a single UTC format across every source
📦 Raw passthrough mode	Set raw_data to true and the aggregator returns the untouched feed structure instead of parsed articles
🔁 Per-feed retries	max_retries controls how many attempts each feed gets, with randomised backoff between them
🆔 Stable entry identity	Each article carries a guid derived from the feed's own identifier, making deduplication across runs reliable
📎 Enclosure extraction	Podcast audio, video and image enclosures are captured with their URL, type and length
🛡️ Resilient parsing	A malformed or partially broken feed is logged and skipped rather than failing the whole run
🔄 Automatic proxy rotation	Each fetch attempt uses a fresh rotating proxy managed by the actor, with no configuration needed from you

---

🚀 Why Choose This RSS Feed Aggregator?

One article shape across every publisher. The whole point of an aggregator is that downstream code should not care whether a source emits RSS 2.0 or Atom, or whether it puts body text in summary or description. Every entry comes back with the same keys, populated from whichever source field was present.

Dates you can actually sort on. Publishers emit timestamps in wildly inconsistent formats. The aggregator normalises publication and update times to ISO 8601 in UTC, which is the difference between a feed collection you can query chronologically and one you have to clean first.

Raw mode when you need the original. Normalisation is lossy by definition. When you are debugging a feed, migrating a legacy pipeline, or need publisher-specific extension elements, raw_data gives you the full channel structure as delivered.

Failures stay contained. A single unreachable or malformed feed does not take the run down with it. Each URL gets its own retry budget, and problems are logged per feed so you can see exactly which source is broken.

---

📥 Input

{
  "rss_feeds": [
    { "url": "https://www.federalreserve.gov/feeds/press_monetary.xml" },
    { "url": "https://www.marketwatch.com/rss/topstories" }
  ],
  "raw_data": false,
  "max_retries": 3
}

🔧 RSS News Aggregator Input Fields

Field	Type	Required	Default	Description
rss_feeds	array	Yes	—	List of RSS feed URLs to fetch and parse. Prefilled with a Federal Reserve monetary policy feed and a MarketWatch top stories feed.
raw_data	boolean	No	false	If enabled, the actor skips field extraction/normalization and returns the raw XML content of each feed instead of parsed articles.
max_retries	integer	No	3	Number of retry attempts for a feed URL before giving up.

💡 Input Examples

Aggregate two financial news feeds with default settings:

{
  "rss_feeds": [
    { "url": "https://www.federalreserve.gov/feeds/press_monetary.xml" },
    { "url": "https://www.marketwatch.com/rss/topstories" }
  ]
}

Fetch raw feed structures for a migration or debugging job:

{
  "rss_feeds": [
    { "url": "https://example.com/blog/atom.xml" }
  ],
  "raw_data": true
}

Monitor several publisher feeds with a higher retry budget:

{
  "rss_feeds": [
    { "url": "https://feeds.example-news.com/technology" },
    { "url": "https://feeds.example-news.com/business" },
    { "url": "https://securityblog.example.org/rss" }
  ],
  "raw_data": false,
  "max_retries": 5
}

---

📤 Output

{
  "source": "federalreserve.gov",
  "feeds": [
    {
      "title": "FRB: Press Release - Monetary Policy",
      "link": "https://www.federalreserve.gov/newsevents/pressreleases/monetary20260729a.htm",
      "description": "Federal Reserve issues FOMC statement",
      "pub_date": "2026-07-29T18:00:00.000Z",
      "guid": "https://www.federalreserve.gov/newsevents/pressreleases/monetary20260729a.htm",
      "author": "",
      "category": "",
      "source": "federalreserve.gov",
      "enclosure": null
    }
  ]
}

🧾 RSS Aggregator Output Fields

Field	Type	Description
source	string | null	Source domain the feed was fetched from.
feeds	array | null	Articles collected from the feed. In raw mode, this contains the feed's raw channel structure instead.

🧾 Fields Inside Each feeds Entry (Parsed Mode)

Field	Type	Description
title	string	Article headline as published in the feed.
link	string	Canonical URL of the article.
description	string	Article summary with HTML markup removed.
pub_date	string	Publication or update time in ISO 8601 UTC format; empty when the feed supplies no date.
guid	string	Unique entry identifier from the feed, falling back to the article link.
author	string	Byline supplied by the feed, if any.
category	string	First category tag on the entry.
source	string	Domain the entry came from.
enclosure	object | null	Attached media as url, type and length, or null when the entry has none.

When raw_data is true, the feeds array holds the feed's channel structure converted from XML to JSON exactly as the publisher delivered it, with no field normalisation applied.

---

💻 How to Use the RSS News Aggregator (Step by Step)

Step 1: Collect the feed URLs you want to monitor

Start by gathering the actual RSS or Atom endpoints, not the human-readable pages. Most publishers link theirs in the page footer or expose it via an application/rss+xml link tag in the HTML head. Paste each one into the rss_feeds array. The input is prefilled with a Federal Reserve press release feed and a MarketWatch top stories feed, which are useful for a first test run.

Step 2: Decide between parsed and raw output

Leave raw_data off for almost every use case — you get consistent, clean article objects with stripped HTML and normalised dates. Switch it on only when you specifically need the publisher's original structure, for example to read namespaced extension elements the normaliser does not carry through, or to diagnose why a feed is producing unexpected entries.

Step 3: Set the retry budget

max_retries controls how many attempts each feed URL gets before the aggregator gives up on it, with randomised backoff between attempts. Three is a sensible default. Raise it for feeds hosted on infrastructure that is slow or intermittently unavailable, and remember that each retry uses a fresh proxy, so a transient block on one attempt often clears on the next.

Step 4: Run the aggregator and watch the per-feed log

Start the run. The log reports each feed URL, the attempts it took, and any HTTP status or parse problem encountered. A feed that logs "did not parse cleanly" with no entries is genuinely malformed at the source; a feed that logs repeated HTTP failures is a fetch problem and usually worth another attempt later.

Step 5: Read the results, one item per feed

Each successfully fetched feed becomes one dataset item, keyed by its source domain, with the articles inside feeds. That structure keeps a source's articles together, which is convenient if you are rendering a per-publisher digest. If you would rather have one row per article, flatten the feeds array when you export.

Step 6: Deduplicate across scheduled runs

RSS feeds overlap heavily between consecutive fetches — the same ten stories appear until they roll off the feed. Use guid as your deduplication key, since it is derived from the publisher's own entry identifier and falls back to the article link. Storing seen GUIDs between runs turns the aggregator into a clean new-articles-only pipeline.

Step 7: Export or push the news data downstream

Export the dataset as JSON to keep the nested article arrays intact, or flatten to CSV for spreadsheet work. For continuous monitoring, schedule the actor and attach a webhook so each completed run pushes fresh articles into your own database, a Slack channel, or a summarisation pipeline.

---

🔌 API Access & Integrations

Fetch and parse feeds in one synchronous call:

curl -X POST "https://api.apify.com/v2/acts/scrapers-hub~rss-news-aggregator/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "rss_feeds": [
      { "url": "https://www.federalreserve.gov/feeds/press_monetary.xml" },
      { "url": "https://www.marketwatch.com/rss/topstories" }
    ],
    "raw_data": false,
    "max_retries": 3
  }'

The same run in Python, flattening the articles as they arrive:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run = client.actor("scrapers-hub/rss-news-aggregator").call(
    run_input={
        "rss_feeds": [
            {"url": "https://www.marketwatch.com/rss/topstories"},
        ],
        "max_retries": 3,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    for article in item.get("feeds") or []:
        print(article["pub_date"], "|", item["source"], "|", article["title"])

The dataset is a standard Apify dataset, so it plugs straight into Zapier, Make, Google Sheets or Slack, and webhooks can fire on run completion to push new articles into your own system.

---

💡 Best Use Cases for Aggregated RSS News Data

📈 Financial and macroeconomic monitoring

Central banks, regulators and financial newswires all publish RSS. Aggregating them and sorting on the normalised pub_date gives you a single chronological tape of policy statements and market news, with source telling you instantly whether an item came from a regulator or a commercial publisher.

🏢 Competitor and industry tracking

Point the aggregator at competitors' blogs and press feeds and run it on a schedule. Matching keywords against title and description surfaces product launches, funding announcements and positioning changes the day they are published rather than the week you happen to check.

🤖 Content pipelines for summarisation and AI

The cleaned description text is exactly what a summarisation or classification model wants — plain text, no markup. Combined with link for the full article and guid for deduplication, it makes a solid ingestion layer for a news digest product or a retrieval pipeline.

🚨 Security advisory and incident feeds

Vendor security advisories and CVE feeds are almost universally distributed as RSS. Aggregating them with a raised max_retries and filtering on category gives a security team a consolidated advisory stream without visiting a dozen vendor portals.

🎙️ Podcast and media cataloguing

Podcast feeds carry their audio in enclosures. The enclosure object exposes the media url, type and length for every episode, which is enough to build a catalogue, mirror a back-catalogue, or drive a media player.

📊 Publication cadence and volume analysis

Because pub_date is normalised across sources, you can measure how often each publisher posts, when in the day they post, and how their volume changes over time. Grouping by source and category turns a feed list into a simple editorial analytics dataset.

🗞️ Internal news digests and dashboards

One dataset item per feed maps neatly onto a per-section digest. Pulling the top few entries from each feeds array by pub_date produces a daily briefing email or a dashboard panel with almost no transformation work.

---

⚙️ Tips for Better RSS Aggregation Results

• Verify each feed URL in a browser first. A URL that returns HTML rather than XML will parse to nothing; the actual feed endpoint is usually linked from the page's head or footer.
• Use guid for deduplication, not title. Headlines get edited after publication, and some publishers reuse titles across entries. The GUID is the publisher's own stable identifier.
• Keep raw_data off unless you need the original XML tree. Parsed mode gives you consistent keys and clean text; raw mode hands you whatever structure the publisher chose, which differs per source.
• Raise max_retries for feeds that are intermittently unavailable. Every attempt uses a fresh proxy, so an extra retry frequently resolves a transient block or timeout.
• Schedule short, frequent runs rather than rare large ones. Feeds only expose a rolling window of recent entries, so a daily run against a fast-moving publisher will silently miss stories.
• Watch for feeds with empty pub_date values. Some publishers omit dates entirely; if chronological ordering matters, decide up front whether to fall back on run time or drop those entries.

---

🛠️ Troubleshooting

One of my feeds produced no item at all. That feed failed every fetch attempt. Check the log for the URL — it records the HTTP status or exception for each try. Confirm the endpoint is still live and returns XML, then raise max_retries if the failures look intermittent rather than permanent.

The feeds array is empty but the item exists. The feed was fetched but contained no parseable entries. This happens with malformed XML and with feeds that are technically valid but currently empty. The log notes when a feed did not parse cleanly, along with the underlying parser exception.

Dates are missing on some articles. pub_date is only populated when the feed supplies a publication or update timestamp. Some publishers omit them entirely, in which case the field comes back as an empty string rather than a fabricated value.

Raw mode output looks completely different per feed. That is expected. With raw_data enabled there is no normalisation at all — you get the publisher's own channel structure converted from XML to JSON, and RSS and Atom feeds have genuinely different shapes.

Article descriptions look truncated. Many publishers deliberately put only a teaser in the feed summary. The aggregator returns what the feed contains after stripping HTML; use the link field to fetch the full article if you need the complete text.

---

❓ Frequently Asked Questions About RSS News Aggregation

What does the RSS News Aggregator actually do? It fetches every feed URL you supply, parses the RSS or Atom XML, and returns one dataset item per feed containing normalised articles with titles, links, cleaned descriptions, ISO 8601 dates, GUIDs, authors, categories and enclosures.

Does the aggregator support Atom feeds as well as RSS? Yes. Both formats are parsed into the same article shape, so you can mix RSS 2.0 and Atom sources freely in one run.

How many RSS feeds can I aggregate in a single run? There is no fixed limit in the input — add as many URLs to rss_feeds as you need. Each is fetched with its own retry budget.

What is the difference between raw mode and parsed mode? Parsed mode normalises every entry into consistent fields with HTML stripped and dates converted. Raw mode skips all of that and returns the feed's original channel structure converted from XML to JSON.

How are publication dates formatted in the output? As ISO 8601 UTC strings, for example 2026-07-29T18:00:00.000Z. Feeds that supply no date produce an empty string.

Is HTML removed from the article summaries? Yes, in parsed mode. The description field contains plain text with markup stripped, which is much easier to feed into search, analytics or language models.

How do I avoid duplicate articles when running the aggregator on a schedule? Deduplicate on the guid field, which comes from the publisher's own entry identifier and falls back to the article link when none is supplied.

Can I extract podcast audio files from a feed? Yes. Where an entry carries an enclosure, the enclosure object returns its url, type and length.

What happens if one feed in my list is broken? Only that feed is affected. It is retried up to max_retries times, logged, and skipped; every other feed in the run is processed normally.

Do I need to configure proxies for the RSS aggregator? No. Proxy rotation is handled automatically, with a fresh proxy session for each fetch attempt.

Why does the output group articles by feed instead of one row per article? Keeping each source's entries together in the feeds array makes per-publisher digests straightforward. If you want one row per article, flatten the array on export.

Can I use this RSS scraper to monitor competitor blogs? Yes — that is one of its most common uses. Schedule regular runs against competitor feeds and match keywords against title and description.

Does the aggregator fetch the full article text from the linked page? No. It returns what the feed publishes, which is often a summary. The link field points to the full article if you need to retrieve it separately.

How do I export aggregated RSS data to Google Sheets or a database? Export the dataset from the console in JSON, CSV or Excel, or connect it through the Apify API, webhooks, Zapier, Make or the Google Sheets integration.

Is aggregating RSS feeds legal? RSS feeds are published specifically to be consumed by aggregators, so fetching them is generally uncontroversial. Republishing full article content, however, is governed by the publisher's copyright and terms — that responsibility is yours.

---

🆘 Support & Feedback

If a particular feed does not parse correctly, open a report on the actor's Issues tab and include the feed URL. Feed quirks are usually publisher-specific, and having the exact URL makes them straightforward to reproduce.

Need something custom — additional normalisation rules, per-article dataset rows, or a direct integration into your content pipeline? Email scraperhubapi@gmail.com with the details.

If the RSS News Aggregator saves you time, a review on the actor page helps other people find it.

---

⚖️ Disclaimer

This RSS aggregator fetches only publicly published feed endpoints, using the same mechanism as any feed reader. It does not bypass paywalls, authentication or access controls.

Article text, headlines and images remain the property of their publishers. You are responsible for how you use aggregated content: republishing full articles, or reproducing content beyond what the publisher's terms and applicable copyright law permit, is your obligation to assess. Where feed content includes personal data such as author names, handle it in line with GDPR, the UK GDPR, CCPA and similar frameworks.

Use of this actor must also comply with each publisher's terms of service and with Apify's platform terms. Fetch at a reasonable frequency and respect any rate limits a publisher states.

For data removal requests relating to content collected by this actor, contact scraperhubapi@gmail.com.