RSS Feed Scraper & News Monitor

Watch any list of RSS, Atom or JSON feeds — or plain site URLs — and get back only the items published since your last run. Optionally with the full article text, not just the 150-character summary the feed carries.

Built for scheduled runs: point it at 50 blogs, run it every hour, and each run delivers the handful of posts that are actually new. You are charged per delivered item, so an hour with nothing new costs you nothing.

Try the tech-news monitoring example, or use the stateless preview below.

Try it with ten items — at most $0.02

Paste this into the JSON input editor and click Start:

{
  "sources": ["https://news.ycombinator.com/rss", "https://blog.cloudflare.com/rss/"],
  "onlyNewItems": false,
  "maxItemsPerFeed": 5,
  "maxItems": 10,
  "includeFullText": false
}

This preview returns up to ten entries without changing your monitoring memory. The charge is $0.002 per delivered entry. Filter out success: false rows before sending the results downstream.

Recipe: a daily competitor-news feed

1. Replace sources with the publishers you follow. Use a small maxItems while testing.
2. Set onlyNewItems: true and choose a distinct stateStoreName, such as my-industry-news. Use the same memory name on subsequent runs.
3. Save as an Apify task and schedule it daily. Export the dataset or pass it to an n8n workflow for summarization.
4. An unchanged feed can return zero rows: this is normal and has no delivered-item charge. If you change keyword filters, use a new memory name to reconsider previously filtered entries.

Do not run tasks that share one memory name at the same time. State is saved at the end of a run; interrupted or overlapping runs can redeliver entries. Entries deferred by a limit are reconsidered while they remain in the publisher's feed; items that fall out of that feed cannot be recovered.

What it does

• Reads every common feed format — RSS 2.0, RSS 1.0 (RDF), Atom and JSON Feed. Measured on 53 live sources: 51 worked (96%).
• Finds the feed for you. Give it https://blog.cloudflare.com or https://stripe.com/blog and it looks for the feed the page declares, then for the usual paths (/feed, /rss.xml, /atom.xml, /index.xml, /feed.json, …).
• Remembers what it already sent. Item IDs are kept in a named key-value store, so a scheduled run returns new items only — previously recorded entries are skipped on later sequential runs. A run with nothing new delivers 0 items and costs 0.
• A per-feed limit throttles, it does not discard. Items over maxItemsPerFeed remain eligible for a following run, even when the feed itself has not changed. The publisher must still include them in its feed.
• Skips unchanged feeds for free. It sends If-None-Match / If-Modified-Since; a 304 Not Modified costs you nothing at all. About 60% of publishers support this (32 of 51 feeds in our measurement).
• Fetches the real article when you ask for it. Feeds usually carry 100–200 characters; the extracted body is around 10× longer (median 686 → 6,700 characters in our measurement).
• Filters before charging. Keyword, exclude-keyword, author and date filters are applied first. Items you filter out are not billed.

Input

Field	Type	What it does
sources	array	Feed URLs or site URLs. Required.
keywords	array	Keep items matching any keyword (case-insensitive).
excludeKeywords	array	Drop items matching any of these.
matchIn	array	Fields the keyword filters look at. Default: title + summary.
authors	array	Keep only these authors.
publishedWithinHours	integer	Drop items older than this. 0 = no limit.
onlyNewItems	boolean	Default on. Deliver only what appeared since the last run.
stateStoreName	string	Named store for the memory, so separate watch lists don't mix.
includeFullText	boolean	Open each article and extract the body. Default off.
fullTextMinChars	integer	Shorter text counts as "not found". Default 500.
includeContentHtml	boolean	Also return HTML, not only plain text.
maxItemsPerFeed, maxItems	integer	Limits. Counted on delivered items.
maxFeedsPerSite	integer	Take more than one feed when a site declares several.
maxConcurrency	integer	Feeds read in parallel. Default 5.
useApifyProxy	boolean	For the few publishers that block datacenter traffic.

Example input

{
  "sources": ["https://techcrunch.com/feed/", "https://blog.cloudflare.com", "https://www.theverge.com/rss/index.xml"],
  "keywords": ["ai", "acquisition"],
  "publishedWithinHours": 24,
  "onlyNewItems": true,
  "includeFullText": true
}

Output

One dataset item per feed entry:

{
  "title": "Cloudflare acquires Human Native",
  "url": "https://blog.cloudflare.com/cloudflare-acquires-human-native/",
  "id": "https://blog.cloudflare.com/cloudflare-acquires-human-native/",
  "author": "Matthew Prince",
  "publishedAt": "2026-09-17T13:00:00.000Z",
  "updatedAt": null,
  "summary": "The web has always run on a bargain…",
  "contentText": "The web has always run on a bargain between…",
  "categories": ["AI", "Acquisitions"],
  "imageUrl": "https://cf-assets.www.cloudflare.com/…/image.png",
  "enclosures": [],
  "commentsUrl": null,
  "feedTitle": "The Cloudflare Blog",
  "feedUrl": "https://blog.cloudflare.com/rss/",
  "feedType": "rss",
  "fullText": "The web has always run on a bargain… (6,700 characters)",
  "wordCount": 1104,
  "success": true,
  "scrapedAt": "2026-09-19T04:12:07.881Z"
}

A source whose feed could not be found or read produces one row instead of failing the run, and that row is not charged:

{
  "input": "https://www.anthropic.com/news",
  "success": false,
  "itemsDelivered": 0,
  "error": "No RSS, Atom or JSON feed found for this URL. If the site has one, pass its feed URL directly."
}

What it cannot do

• It cannot invent a feed that doesn't exist. Sites that publish no RSS/Atom/JSON feed (for example anthropic.com/news) return a failure row. It does not scrape article lists out of HTML.
• Full article text is not guaranteed. Measured at 84–88% of articles. It fails on pages rendered entirely by JavaScript, on paywalled articles (NYTimes, The Economist), and on pages that aren't articles at all (a landing page, a forum post). Those rows carry fullText: null and a reason; the item is still delivered and charged, because the feed entry itself is the product.
• Short posts fall below the length floor. Link-blog entries of 200–400 characters are treated as "no body" at the default fullTextMinChars: 500. Lower it if you watch link blogs.
• Some publishers block datacenter IPs outright. AP News returns 403 to any datacenter request. Turn on Apify Proxy for those.
• No login, no anti-bot bypass. Feeds behind a subscriber login are out of scope.
• Feeds only carry what the publisher puts in them. Missing authors, missing dates and truncated summaries are the publisher's choice, not a bug here. Dates are normalised to ISO 8601 when present.
• It is not a full-site archive. A feed typically holds the latest 10–50 items; the actor cannot reach back further than the feed goes.

Pricing

Pay per event: $2.00 per 1,000 delivered items. Platform usage (compute, proxy, storage) is included — you pay for delivered items and nothing else.

Not charged: filtered-out items, items already delivered in an earlier run, unchanged feeds (304), and sources that failed.

Tips

• Schedule it hourly with onlyNewItems: true and publishedWithinHours: 48. The date filter guards against a feed that republishes old items with new IDs.
• Use stateStoreName to separate watch lists — "competitors" and "industry-news" won't hide each other's items.
• Leave includeFullText off for alerting and turn it on for summarising. Off, a 50-feed run finishes in a few seconds.
• Chain it into an LLM step with the text output link: it returns title, URL, date and body only.

Release validation

The 2026-09-24 regression check read 51 of 53 real feed/site inputs (96.2%), delivering 153 entries with a three-item per-feed limit. The unavailable inputs were AP News and a site without a discovered feed. On 2026-09-26 the prefilled preview returned ten entries, and two sequential runs of the monitoring example returned ten entries each with no overlapping article URLs. These checks do not guarantee every publisher remains available.