RSS Feed Reader with New-Item Detection

Read RSS 2.0 and JSON Feed into structured item rows for alerts and content pipelines, with a status row for each feed. Omit stateStoreName for a one-off read using the run's own storage. For recurring monitoring, supply a named history store and reuse it to output only previously unseen item IDs across runs.

Get titles, links, publication dates, authors, plain-text summaries, HTML content, categories and enclosures when supplied by the publisher. The actor reads feed content; it does not fetch linked articles or create AI summaries. It detects new IDs, so edits to an existing item do not trigger an update event.

Start a watch list

Run with a feed URL and an explicit history-store name:

{
  "feedUrls": ["https://www.jsonfeed.org/feed.json"],
  "stateStoreName": "my-feed-watch",
  "onlyNewItems": true,
  "maxItemsPerFeed": 100
}

The named store enables history between runs. If you omit stateStoreName, the actor uses the run's default key-value store and does not reuse history from previous runs. With that default, onlyNewItems: true still treats encountered IDs as new on each new run; the output-position limit still applies.

Reuse both the URL and store name on later runs. Store names use lowercase letters, digits and internal dashes, up to 63 characters; store IDs are not accepted. The supplied name requests read and write access to that store, including existing history.

With a persistent store, every encountered ID counts as new on the first successful read of a URL that has no saved history. For alerts about later posts, use that read as a baseline and start delivering alerts afterward. Configure scheduling and delivery separately in Apify or your automation. Do not overlap runs that use the same named store for the same feeds: history updates are not coordinated between runs.

Use the output

The dataset contains three row types. Filter by type before using rows in an automation. Dataset views select columns; they do not filter out other row types.

type	Contents
item	Feed content, itemId, idSource, isNew, firstSeenAt, position
feed	URL, ok, status, HTTP result, counts and errors
summary	Run totals, status counts, storage mode and URL-cap flag

The summary reports stateStore: null and historyPersistent: false when the store name is omitted. With an explicit named store, stateStore contains that name and historyPersistent is true. These fields identify the selected storage mode; check each feed's status to confirm whether its history was saved successfully.

Example output

These selected fields come from a saved read of JSON Feed's public feed. The item is usable because its matching feed row has ok: true; the summary reports no failed feeds.

[
  {
    "type": "item",
    "feedUrl": "https://www.jsonfeed.org/feed.json",
    "title": "Announcing JSON Feed",
    "isNew": true
  },
  {
    "type": "feed",
    "feedUrl": "https://www.jsonfeed.org/feed.json",
    "ok": true,
    "status": "ok"
  },
  {
    "type": "summary",
    "feedsFailed": 0
  }
]

To select usable items from a completed run's JSON dataset export, match each item's feedUrl to a successful feed row. This example reads the entire export before selecting items, because feed status can be written after item rows.

import json

with open("dataset.json", encoding="utf-8") as source:
    rows = json.load(source)

successful_feeds = {
    row["feedUrl"] for row in rows
    if row.get("type") == "feed" and row.get("ok") is True
}
items = [
    row for row in rows
    if row.get("type") == "item"
    and row.get("feedUrl") in successful_feeds
]

The run's default key-value store also contains all feed status rows in FEEDS and the summary in OUTPUT. A successful actor run can contain failed feeds: check feedsFailed and the feed rows. Match item rows to a feed row with ok: true before delivering them. A history-write failure can leave item rows in the dataset without saving their IDs; those items can appear as new again later.

A successful repeat using persistent history can have no item rows. The actor can reuse saved HTTP validators for that feed, and HTTP 304 gives status: "not-modified" with no item rows even when onlyNewItems: false; it does not rebuild a full snapshot from cached content. Omitting stateStoreName starts without validators or seen IDs from previous runs.

Options

Field	Purpose	Default
feedUrls	Feed URLs or pages declaring a feed; aliases include urls, startUrls, feeds, url	Required
stateStoreName	Named store for history between runs; omit for run-local storage	Omitted
onlyNewItems	Emit only unseen IDs within the position limit; cross-run detection requires a named store	false
maxItemsPerFeed	First 1 to 10,000 feed positions eligible for output	100
resetState	Ignore earlier history for supplied feeds and replace it after successful reads	false
includeContent	Include contentHtml; plain-text summary remains when disabled	true
discoverFeeds	Follow the first feed link declared by an HTML page	true
respectRobotsTxt	Check robots.txt before feed requests and redirects	true
maxConcurrency	Process 1 to 20 feeds concurrently, subject to host pacing	10
timeoutSecs	Per-request timeout, from 5 to 120 seconds	30

Supply a store name explicitly for recurring monitoring. Invalid supplied names are rejected. If the selected store cannot be opened, the run stops without switching histories or substituting run-local storage. Individual history read or write failures produce internal-error feed rows while other feeds continue. Omitting the name selects run-local storage from the start; it is not a fallback for named-store failures.

Limits

• Up to 1,000 normalized, distinct URLs are accepted. Extra URLs are dropped and feedCapReached is set. This is an input cap, not a completion or throughput guarantee. Malformed URL strings are discarded without a feed row; no usable URL makes the run fail.
• maxItemsPerFeed limits positions before filtering for new IDs. IDs beyond that limit are still remembered and counted as new after a successful history write. With persistent history, raising the limit on a later run will not make those IDs new again.
• Identity uses the publisher's GUID or ID, then the item link, then a title-and-date hash. ID changes can repeat an article. In a named history store, a homepage URL and its direct feed URL maintain separate histories.
• History retains all IDs in the current feed plus earlier IDs within a target of 5,000 per URL. Larger current feeds can exceed that target. With persistent history, an old ID pruned from that history can count as new when it returns.
• Same-host requests run serially with a one-second gap after each response. Different hostnames and separate runs do not share pacing. Repeated HTTP 429, 503 or network failures trigger backoff and a stop for that host within the process.
• Responses are capped at 15 MB after decompression, with at most five redirects. Truncated XML may retain completed items with ok-with-errors; inspect parse errors. Nonstandard dates may be null.
• HTML content is publisher-supplied and is not sanitized for display. Login pages, paywalls and bot checks are not bypassed. Discovery follows a declared feed link; it does not search every possible feed location.
• Real-source release validation covers RSS 2.0 and JSON Feed. Atom, RDF and older RSS parser paths have bundled-example checks, without retained real-source validation for this release.

Check publisher terms and your right to reuse content. Robots.txt checking does not grant permission or guarantee compliance. Disable it only for feeds you own or have permission to read.

Cost

No developer rental or per-result fee. Apify platform usage and storage charges may still apply under your account's plan. There is no measured cost-per-1,000-feeds estimate for this release.

Privacy and support

Feed content, publisher-supplied author names, feed status and summaries are saved in your Apify account. History records contain feed URLs, HTTP validators, encountered IDs and first-seen times. With an explicit stateStoreName, these records are reused from the named store between runs. With the name omitted, they are stored in the run's default key-value store and are not reused automatically by later runs. Run-local storage does not mean automatic deletion: the actor does not delete outputs or expire history by age. Your Apify retention and sharing settings govern stored data. Requests disclose requested URLs to publisher and redirect servers; avoid URLs containing private tokens. resetState does not delete previous run outputs or unrelated history.

Built and maintained by AI agents for Madrasco. Use the actor's Issues tab or support@madrasco.dev for support. Platform replies may be AI-assisted; a human owner can be reached on request.