RSS Feed Merger & Smart Deduplicator

Merge up to 100 public RSS 2.0, Atom, and RDF/RSS 1.0 feeds into one normalized dataset. Filter by date or keywords, remove repeat coverage deterministically, preserve source provenance, and export the result from Apify as JSON, CSV, Excel, XML, or RSS-compatible downstream data.

What this Actor does

• fetches only the public feed URLs you provide
• supports RSS 2.0, Atom, and RDF/RSS 1.0 XML
• normalizes titles, URLs, dates, authors, categories, enclosures, and short summaries
• removes common tracking parameters from deduplication URLs
• merges repeated items across feeds while preserving every source-feed URL
• filters by include keywords, exclude keywords, and publication date
• sorts dated rows newest first
• writes one row per unique item and a run summary to OUTPUT

The Actor does not crawl linked articles. It uses feed-provided metadata and limits summary text to 500 characters.

Input

{
  "feedUrls": [
    "https://feeds.npr.org/1001/rss.xml",
    "https://feeds.bbci.co.uk/news/rss.xml"
  ],
  "maxItemsPerFeed": 100,
  "maxTotalItems": 1000,
  "deduplicateBy": "smart",
  "includeKeywords": [],
  "excludeKeywords": [],
  "publishedAfter": "",
  "fetchTimeoutSecs": 15
}

smart deduplication uses the normalized article URL first, then the RSS GUID or Atom ID, then a normalized title plus publication date. Choose canonical-url, guid, or title when one field should control matching. Missing selected fields use a unique fallback instead of collapsing unrelated rows.

Output

Each dataset row includes:

• title and normalized article URL
• publication and update timestamps
• author, categories, short summary, and enclosure metadata
• original feed URL, feed title, and format
• deterministic deduplication key
• number of occurrences merged
• every feed URL that supplied the duplicate item
• merge timestamp

The OUTPUT record reports feed successes and failures, parsed and filtered items, duplicates removed, emitted rows, and truncation state.

Pricing

Pay per event:

• $0.00005 per Actor start
• $0.001 per unique item emitted ($1 per 1,000 unique items)

Failed feeds, filtered rows, and duplicates removed before output are not charged as unique items.

Standards

• RSS 2.0 specification
• RFC 4287 — The Atom Syndication Format

Network and content safety

The Actor rejects credentials, private/local/reserved IP space, non-standard ports, oversized responses, redirect loops, and XML containing DTD or entity declarations. DNS is resolved before each request, only a public address is accepted, and the request is pinned to that address while HTTPS certificate validation still uses the requested hostname. Every redirect is revalidated.

Only feed XML is fetched. Linked article pages, images, audio, and video are not downloaded. Output is feed-provided metadata, not a license to republish source content. Follow each publisher's terms, attribution rules, and applicable law.

Boundaries

Deduplication is deterministic but heuristic. Review the dedupe key and source list before using the output to delete records or make compliance, financial, legal, medical, or safety-critical decisions. Feed availability and metadata can change between runs.