RSS & Atom Delta API

Detect only new and updated feed items across Actor runs. Supply public RSS 2.0, Atom, or JSON Feed 1.1 URLs; the first run creates a silent baseline, and later runs emit only changes.

Why use it

Most feed readers return the whole feed every time. This Actor keeps bounded per-monitor state, uses conditional HTTP requests, and emits table-ready delta records for automations, agents, and webhooks.

Input

{
  "feedUrls": [
    "https://hnrss.org/newest"
  ],
  "monitorId": "product-watch",
  "resetState": false,
  "maxFeedBytes": 1000000,
  "maxRedirects": 3,
  "maxStateItems": 5000
}

Field	Type	Default	Limits	Meaning
feedUrls	string[]	required	1–10	Explicit public HTTP(S) feed URLs.
monitorId	string	default	1–64 chars	Stable namespace for independent monitors; letters, numbers, _, and - only.
resetState	boolean	false	—	Delete prior state and silently rebaseline.
maxFeedBytes	integer	1000000	100 KB–2 MB	Per-feed response-body limit.
maxRedirects	integer	3	0–5	Redirect limit; every destination is revalidated.
maxStateItems	integer	5000	100–10,000	Retained item hashes per feed.

Undocumented input fields are rejected.

Output

The default dataset receives one row per delivered change:

{
  "monitorId": "product-watch",
  "feedUrl": "https://example.com/feed.xml",
  "feedTitle": "Product releases",
  "changeType": "new",
  "itemId": "release-42",
  "itemTitle": "Version 4.2",
  "itemUrl": "https://example.com/releases/42",
  "publishedAt": "2026-07-19T07:00:00Z",
  "updatedAt": null,
  "summary": "Bounded, sanitized text",
  "item": {
    "id": "release-42",
    "title": "Version 4.2",
    "url": "https://example.com/releases/42"
  }
}

The default key-value store record OUTPUT contains the run summary (feedsSucceeded, feedsFailed, changesEmitted, and secret-safe error types).

Pay-per-event pricing

Event	Proposed price	Charged when
feed-checked	$0.0020	A feed was fetched or conditionally confirmed unchanged and successfully processed.
change-detected	$0.0002	A new/updated item was precommitted and submitted through Apify's paid dataset-delivery path. Successful calls report only the SDK-fulfilled count; the at-most-once failure tradeoff below still applies.
apify-actor-start	$0.00005	Synthetic Apify start event; platform-managed, never charged by Actor code.

Failed fetches, unsafe URLs, oversized bodies, and parse failures are not charged by Actor code. Before paid output, the Actor preflights the remaining event budget and commits exactly the chargeable prefix to state. If state persistence fails, no feed or change event is charged and no dataset row is written. If the run's event budget is exhausted, remaining network work stops; a partially chargeable batch commits and delivers only its chargeable prefix, while the rest remains retryable with older conditional headers.

Paid output uses at-most-once semantics, not exactly-once transactions across Apify storage services. If billing or dataset delivery fails after the state precommit, the run fails and that committed prefix is not rolled back or automatically offered again; this prevents duplicate customer charges and duplicate rows on retry but can omit that prefix after a rare platform failure. Use the failed-run signal for operational alerting.

The controlled 20-case benchmark is in benchmark-results/ECONOMIC_BENCHMARK.md. It demonstrates positive margin potential under the stated model; it is not a claim of cloud payouts or external usage.

Safety and scope

• Only explicit public http:// and https:// feed URLs are accepted.
• DNS results, redirects, and destination IP classes are validated.
• Connections are pinned to a validated address while retaining the original hostname for TLS SNI and certificate checks.
• Private, loopback, link-local, reserved, metadata-service, and otherwise unsafe destinations are rejected.
• The fetcher applies a 60-second monotonic budget across URL validation, redirects, response headers, and chunked body reads. The Actor also stops awaiting the off-loop worker after 61 seconds, so a stuck system resolver cannot block lease renewal or batch termination. Responses, redirects, summaries, feed count, and retained state are also bounded.
• RSS/Atom XML is parsed with defusedxml.
• Active HTML is removed from titles and summaries.
• Item/article URLs found inside feed entries are never requested. Retrieval is limited to each explicit feed URL and its separately revalidated HTTP redirects; a feed operator can redirect that feed request to another public HTTP resource, whose body must still pass feed parsing before any state or charge is produced.
• Authentication bypass, CAPTCHA bypass, paywall bypass, and private-data collection are out of scope.
• The feed operator's terms and applicable laws remain the user's responsibility.

State semantics

State is kept in the named key-value store rss-atom-delta-state-v1. A state key is derived from the exact feed URL plus monitorId. The first successful run emits no historical items. Only the first maxStateItems unique entries in source order are evaluated and retained; overflow entries are ignored rather than repeatedly labeled and billed as new. JSON Feed items without required IDs are discarded rather than assigned unstable identifiers.

On Apify cloud, runs sharing a monitorId use the public Request Queue list_and_lock_head API before state is read, with a unique client key and 180-second lock renewed every 60 seconds. A second overlapping cloud run for that monitor fails instead of racing; renewal loss cancels the batch, owner-scoped unlock releases a successful lease, and a crashed process relies on platform lock expiry. Different monitor IDs remain independent. Crawlee's local file-backed Request Queue did not serialize separate development containers in verification, so local multi-process overlap is unsupported and must be avoided; authenticated cloud overlap and renewal verification remain publication gates.

The Actor definition pins the default and minimum run memory to 256 MB, matching the controlled economic benchmark. Callers may choose a larger supported memory allocation; doing so increases compute cost beyond the benchmark model.

Local verification

python -m pip install -r requirements.txt
PYTHONPATH=src python -m pytest tests -q
PYTHONPATH=src python benchmarks/run_benchmark.py
docker build -t rss-atom-delta-api:local .

Current verified checkpoints are recorded in the mission evidence; rerun them after any code change.