RSS Feed Aggregator — Full Article Content Extraction

Aggregate any number of RSS/Atom feeds in one run and get the full article text, not just the feed summary. Every item is fetched, extracted with trafilatura (high-precision main-content extraction), deduplicated, and written as clean, structured JSON.

Ideal for news monitoring, content curation, AI/RAG pipelines, media intelligence, and SEO research — one scheduled run replaces a self-hosted feed reader plus a scraping stack.

What it does

1. Parses every RSS/Atom feed you give it (news sites, blogs, Substack, Google News queries, podcast feeds — anything with a valid feed).
2. Optionally visits each article URL and extracts the complete readable text, author, date, and site name — stripping navigation, ads, and boilerplate.
3. Deduplicates across feeds by URL or title, so overlapping sources don't produce duplicate rows.
4. Filters by publish date (ISO 8601) and/or feed categories.
5. Pushes one structured item per article to the dataset, ready for export as JSON, CSV, or Excel.

Input

{
  "feedUrls": [
    { "url": "https://hnrss.org/frontpage" },
    { "url": "https://techcrunch.com/feed/" }
  ],
  "maxArticles": 100,
  "extractFullContent": true,
  "deduplicateBy": "link",
  "dateFilter": "2026-07-01"
}

Field	Type	Default	Description
feedUrls	array	required	RSS/Atom feed URLs to aggregate
maxArticles	integer	100	Maximum articles per run (0 = unlimited)
extractFullContent	boolean	true	Visit each article and extract the full text
deduplicateBy	string	"link"	link, title, or none
dateFilter	string	""	Only keep articles published after this ISO 8601 date
includeCategories	string	""	Comma-separated category filter (matches feed categories)
proxyConfiguration	object	Apify proxy	Proxy settings used for article fetching

Output

One dataset item per article:

{
  "title": "Show HN: I built a local-first RSS reader",
  "link": "https://example.com/local-first-rss",
  "guid": "https://example.com/local-first-rss",
  "pubDate": "2026-07-16T08:30:00Z",
  "source": "Hacker News: Front Page",
  "sourceUrl": "https://hnrss.org/frontpage",
  "author": "jane_doe",
  "summary": "A local-first RSS reader with offline sync…",
  "fullText": "For the last six months I've been building…",
  "wordCount": 1240,
  "extractionMethod": "trafilatura",
  "categories": ["programming"],
  "tags": ["rss", "local-first"],
  "enclosures": []
}

All fields are null-tolerant: if an article page can't be fetched you still get the title, link, date, and RSS summary.

Use cases

• News monitoring — merge dozens of sources into one deduplicated stream; schedule hourly and pipe new items to Slack, email, or a webhook with an Apify integration.
• AI training / RAG pipelines — fullText + wordCount give you clean, boilerplate-free text chunks with provenance metadata.
• Content curation — feed newsletters or "best of" pages from niche blogs without opening a feed reader.
• Media intelligence — track competitor blogs and industry publications; filter with dateFilter to process only what's new since the last run.
• SEO research — monitor publications in your niche for content gaps and backlink opportunities.

Run it on a schedule

Create an Apify Schedule (e.g. every hour) pointing at this actor with your feed list, and set dateFilter to roll forward — each run then returns only fresh articles. Combine with the dataset API to pull new rows into your own system:

curl "https://api.apify.com/v2/acts/darknezz~rss-feed-aggregator/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -X POST -H "Content-Type: application/json" \
  -d '{"feedUrls": [{"url": "https://hnrss.org/frontpage"}], "maxArticles": 50}'

Pricing

Pay-per-event: $0.01 per article extracted, plus Apify's standard per-run and per-dataset-item micro-fees. 100 fully extracted articles ≈ $1. No subscription, no monthly minimum.

FAQ

Does it work with Atom feeds / podcast feeds? Yes — anything feedparser understands: RSS 0.9x/1.0/2.0, Atom, RDF. Podcast enclosures land in enclosures.

What if a site blocks full-text extraction? You still get the feed-level data (title, link, summary, date). The item's fullText is null and extractionMethod reflects the fallback — the run does not fail.

Can I aggregate Google News search results? Yes — use a Google News RSS URL (https://news.google.com/rss/search?q=your+query) as a feed.

How is this different from a feed reader? A reader shows you summaries in an app. This actor gives you the complete article text as structured data you can pipe into a database, LLM, or BI tool.

How many feeds can I pass? No hard limit — runs process feeds sequentially; use maxArticles to cap cost on large lists.