RSS Feed Reader & Full-Text News Article Extractor

An RSS feed scraper and news article extractor that reads RSS, Atom and JSON feeds, auto-discovers feeds from any website, and returns full-text articles as clean text and Markdown. Use it for content monitoring, news aggregation, media tracking, newsletter research and AI / LLM / RAG pipelines: one dataset row per article with title, URL, ISO publish date, author, categories, summary, full text, Markdown, word count, lead image, language and canonical URL. HTTP only (no headless browser), so it is fast and cheap.

Use cases

• Full-text RSS: many feeds only carry a teaser; this fetches the article and extracts the whole text.
• Brand, competitor and keyword monitoring: schedule it with sinceDate and keywords to get only new, relevant items.
• Feed articles into LLMs, vector databases and summarizers as clean Markdown.
• Build news datasets, research corpora or a personal reader.

Input

Field	Description
feedUrls	RSS, Atom or JSON Feed URLs.
siteUrls	Website URLs; the feed is auto-discovered via <link rel="alternate"> or common paths (/feed, /rss.xml, /atom.xml, ...).
articleUrls	Individual article URLs to extract directly.
maxItemsPerFeed	Newest N items per feed (default 5).
maxItems	Hard cap on items output and charged per run (default 500).
extractFullText	Fetch each article and extract text + Markdown (default on). Off = feed data only, cheaper.
sinceDate	ISO date (2026-10-01) or relative (7 days). Older items are skipped.
keywords	Keep items whose title, summary or categories match any keyword.
maxConcurrency, respectRobotsTxt	Politeness controls.

An empty input {} runs with https://hnrss.org/frontpage, 5 items.

{ "feedUrls": ["https://hnrss.org/frontpage"], "siteUrls": ["https://www.theverge.com"], "maxItemsPerFeed": 10, "sinceDate": "3 days", "keywords": ["AI", "openai"], "extractFullText": true }

Output (one row per item, deduplicated by URL)

{
  "feedTitle": "Example Blog", "feedUrl": "https://example.com/feed", "title": "Hello world",
  "url": "https://example.com/hello", "canonicalUrl": "https://example.com/hello",
  "publishedAt": "2026-10-09T14:00:00.000Z", "author": "Jane Doe", "categories": ["news"],
  "summary": "First 500 characters...", "leadImage": "https://example.com/hero.jpg", "language": "en",
  "wordCount": 842, "text": "Clean article text...", "markdown": "# Hello world\n...",
  "fullTextExtracted": true, "paywalled": false, "note": null, "fetchedAt": "2026-10-10T12:00:00.000Z"
}

A summary (items, extraction failures, robots blocks, paywalls, skipped counts) is saved to the key-value store as OUTPUT.

Pricing (pay per event)

• $0.002 per article with full text extracted (article-extracted): $2 per 1,000.
• $0.0008 per feed item without full text (feed-item): $0.80 per 1,000. Also used when an article cannot be fetched or extracted, so you never pay the higher price for a missing body.
• One event per output item. Set a max charge per run and the actor stops at the limit.

Limitations

• Needs a public RSS, Atom or JSON feed, or a site with a discoverable feed (link rel=alternate, or common paths such as /feed and /rss.xml; up to 3 discovered feeds). It cannot list articles from a site with no feed.
• Defaults to 5 items per feed (maxItemsPerFeed) and 500 items per run (maxItems). Feeds usually expose only their latest entries, so it is not an archive crawler.
• Full-text extraction uses Readability on static HTML only: no JavaScript rendering, no login, no paywall bypass. Pages under 30 extracted words count as failures and return feed data only (charged as a feed item, not an article).
• Fetches wait at least 200 ms between requests to the same host, run up to 4 in parallel (max 10), and time out after 30 s with up to 2 retries. Responses over 15 MB are skipped. robots.txt is honored for article fetches by default.
• Language and lead image come from page metadata and may be empty.

FAQ

Does it bypass paywalls? No. It only reads publicly published feeds and pages. If a page looks paywalled, you get whatever is public plus paywalled: true and a note. Does it respect robots.txt? Yes, for article fetches (switchable, but leave it on). Feed URLs you supply are fetched directly. Requests are rate-limited per host and identify as WintersWorksFeedReader. JavaScript-only sites? Not supported (no headless browser); extraction fails gracefully and you get feed data. Why is some text noisy? Extraction uses Mozilla Readability heuristics; pages like Wikipedia or index pages can include infoboxes or tables. Quality is best on blogs and news articles. Is this legal? It reads publicly published feeds and pages, respects robots.txt by default and does not bypass paywalls or logins. Feed and article content is still the publisher's copyrighted work. Personal analysis, search and monitoring are common uses, but republishing it may need permission. You are responsible for how you use and republish it, and for following each publisher's terms.

Built by Winters Works.

Use from code and AI agents

Call it from any language with the Apify API, or let an AI agent find and run it through the Apify MCP server.

from apify_client import ApifyClient
run = ApifyClient("<YOUR_APIFY_TOKEN>").actor("wintersworks/rss-feed-news-article-extractor").call(run_input={"feedUrls": ["https://hnrss.org/frontpage"]})
items = ApifyClient("<YOUR_APIFY_TOKEN>").dataset(run["defaultDatasetId"]).list_items().items

Results land in the run's dataset (JSON, CSV, Excel). Set a maximum charge per run in the run options to cap spend.