RSS and Atom Feed Monitor

Watch publisher feeds and keep the items you have not already saved. This Actor reads RSS, Atom, and JSON Feed documents, can find a feed from a site URL, and normalizes each entry to the same fields: feed, title, link, author, published time, summary, plain-text content, categories, and guid.

It is built for newsletter editors, research agents, and content teams who want a feed inbox rather than a one-off dump. You pay when a feed is parsed and when an item is saved. There is no start fee.

What you can do with it

• Pull the latest items from a list of feeds into the Apify dataset, then into Sheets, Slack, or your own API.
• Point it at a site homepage and let it follow link rel="alternate" tags (RSS, Atom, or JSON Feed).
• Keep a keyword watch: include keywords are OR, exclude keywords drop a match.
• Schedule it with New items only so the next run skips guids this Actor already delivered for that state namespace.
• Cut a feed off at a date with Published since.

The default input reads the NASA news feed and returns a handful of items. That run does not need a key, a proxy, or a login, and it is the input the Apify daily test uses.

How a run behaves

1. Site URLs are checked against robots.txt. A disallowed page is skipped and listed in the run summary.
2. Feed URLs you typed are also checked. A disallowed feed is not fetched.
3. Each feed that parses is one feed-checked event. A URL that fails is not billed.
4. Items are filtered, newest dated items first, then capped per feed and for the whole run.
5. Each saved item is one new-item event. The guid is stored in a named key-value store (rss-feed-monitor-seen) so a later run can skip it. Article text is not stored in that state.

Crawl-delay in robots.txt is honored up to 10 seconds so one host cannot use the whole run. The client sends a descriptive MonitorActors user agent, retries 429 and 5xx responses with backoff, and waits between requests to the same host. It does not rotate proxies or pretend to be a browser.

Input

{
  "feedUrls": ["https://www.nasa.gov/feed/"],
  "siteUrls": [],
  "includeKeywords": ["mars"],
  "excludeKeywords": ["podcast"],
  "since": "2026-10-01",
  "newItemsOnly": true,
  "stateNamespace": "space-desk",
  "maxItems": 25,
  "maxItemsPerFeed": 10
}

Field	Required	What it does
Feed URLs	No, default is the NASA feed	Direct RSS, Atom, or JSON Feed URLs
Site URLs	No	HTML pages to scan for feed autodiscovery links. Up to three feeds per page
Include keywords	No	Keep items that contain at least one keyword
Exclude keywords	No	Drop items that contain any keyword
Published since	No	ISO date or datetime. Undated items are dropped when this is set
New items only	No, default off	Skip guids already saved for this namespace
State namespace	No, default default	Separate monitors in one Apify account
Maximum items	No, default 5	Run-wide cap
Maximum items per feed	No, default 20	Per-feed cap, newest first
Maximum charge (USD)	No	Extra cap. The run's Apify maximum charge still applies, and the lower one wins

Leave New items only off for a one-shot export. Turn it on for a schedule. A scheduled run that finds nothing new finishes successfully with an empty dataset and a status message, so a quiet hour is not a failure. The first run of a new namespace has an empty memory, so it saves whatever matches.

Output

One dataset item per feed entry:

{
  "feed": "https://www.nasa.gov/feed/",
  "feedTitle": "NASA",
  "title": "NASA example headline",
  "link": "https://www.nasa.gov/news-release/example/",
  "author": "NASA",
  "published": "2026-10-22T15:04:00Z",
  "summary": "Short plain-text excerpt.",
  "contentText": "Plain text of the entry. HTML tags are removed.",
  "contentTruncated": false,
  "categories": ["Mars"],
  "guid": "https://www.nasa.gov/?p=12345"
}

Bodies longer than 20,000 characters are truncated and contentTruncated is true. Summaries are capped at about 1,200 characters.

The key-value record SUMMARY reports items saved, event counts, an estimated charge, feeds checked, and per-URL errors. A run with at least one saved item succeeds even if another feed failed. A run that was not in new-items mode and saved nothing fails, so a broken feed does not look like a healthy empty export.

Pricing

Pay per event. Do not enable the synthetic apify-default-dataset-item event. This Actor charges its own events with Actor.charge and would double-bill if that synthetic event has a price. In the Actor's monetization settings, remove it or set its price to $0, and copy the events below. new-item is the primary event.

Event	Price	When it fires
apify-actor-start	$0.005	Once per run, by the platform (memory is capped at 1 GB, so once). The Actor does not call Actor.charge for it.
feed-checked	$0.001	A feed URL was fetched and parsed
new-item	$0.0005	A normalized item was saved

Twenty feeds and 100 new items cost $0.005 + 20 × $0.001 + 100 × $0.0005 = $0.075. Platform usage is not billed to you; the events above are the whole price. Set a maximum charge on the run. When the next event would pass that cap, the Actor stops, keeps what it already saved, and records stoppedReason: max_charge.

Failed feed URLs are not charged. Items filtered out by keywords, dates, or the new-item memory are not charged.

FAQ

Does it only return new items? Only when you turn on New items only. The memory is per Apify account and per state namespace, in the named store rss-feed-monitor-seen. Deleting that store, or changing the namespace, makes the next run treat items as new.

What if I raise the item cap later? Only guids that were actually saved are remembered. Items left behind by the cap can show up on a later run. Newest dated items are saved first.

Will it scrape the article page? No. It reads the feed document the publisher already offers and returns the text inside that document. It does not open each article URL.

What about robots.txt? Disallow rules are followed for site pages and for feed URLs. If the feed you care about is disallowed, the summary says so and that URL is skipped.

JSON Feed? Yes, when the response is JSON and has a jsonfeed.org version with an items array, or a JSON content type with an items array.

Can I run several monitors? Yes. Give each schedule its own state namespace and its own keyword list. They can share the Actor.

Limits

• This is a feed reader, not a site crawler. Paywalled or truncated feed bodies stay truncated.
• Autodiscovery reads link rel="alternate" tags only. It does not guess /feed paths.
• At most three discovered feeds are taken from one site page.
• Very large feeds over 8 MB are rejected.
• Seen-id lists keep the newest 10,000 guids per feed namespace.
• Platform usage (memory and time) is not billed to you; you pay only the events above.

Run it locally

From this directory, with monitor_core vendored beside the Actor package (see the repository README):

mkdir -p storage/key_value_stores/default
printf '%s\n' '{"feedUrls":["https://www.nasa.gov/feed/"],"maxItems":3}' > storage/key_value_stores/default/INPUT.json
python -m rss_feed_monitor

Dataset items are written under storage/datasets/default/. The summary is storage/key_value_stores/default/SUMMARY.json.

Publish notes

Keep this Actor on limited permissions. It only reads its input and writes its own default storages plus the named seen-id store it creates. Copy .actor/pay_per_event.json into the pay-per-event monetization form, and do not put a price on apify-default-dataset-item.