RSS & Atom Feed Reader + Change Monitor

Parse public RSS, Atom, and RDF feeds into clean structured JSON, then monitor item changes and feed health across scheduled runs.

Technical name: rss-atom-feed-reader-change-monitor

Why use this Actor?

This Actor combines a fast feed reader with persistent monitoring. It is useful when you want both a current feed snapshot and a low-noise stream of changes for automation, AI agents, research, content monitoring, or alerts.

Key capabilities:

• RSS 2.x parsing
• Atom 1.0 parsing
• Atom 0.3 compatibility for common legacy fields
• RSS 1.0 / RDF parsing
• batch processing for many feeds
• normalized item titles, summaries, content, dates, categories, and enclosures
• AI-ready combined plain text
• stable feed and item IDs
• NEW_ITEM, UPDATED_ITEM, and UNCHANGED monitoring
• FEED_ERROR and FEED_RECOVERED health events
• ETag / Last-Modified conditional requests in changes-only monitoring mode
• baseline-first scheduling without flooding the dataset with old feed entries
• keyword and regex filters
• persistent per-feed monitoring state
• SSRF/private-network blocking, including redirects to private addresses
• bounded retries, download limits, item limits, output-size caps, and state limits
• best-effort repair for common malformed XML entities after strict parsing fails

The Actor intentionally does not emit REMOVED_ITEM when an entry disappears from a feed. Many feeds are rolling windows that only expose the newest N entries, so treating disappearance as deletion would create false alerts.

Quick start

Read one feed

{
  "feedUrls": [
    "https://www.nasa.gov/feed/"
  ],
  "maxItemsPerFeed": 25
}

Read multiple feeds

{
  "feedUrls": [
    "https://example.com/news.xml",
    "https://example.org/feed.atom"
  ],
  "maxItemsPerFeed": 100,
  "concurrency": 10
}

Monitor only new or updated items

First run:

{
  "feedUrls": [
    "https://www.nasa.gov/feed/"
  ],
  "monitorMode": true,
  "emitChangesOnly": true,
  "baselineOnly": true,
  "monitorKey": "nasa-news"
}

The first run stores the current entries as a baseline without emitting them.

Later scheduled runs:

{
  "feedUrls": [
    "https://www.nasa.gov/feed/"
  ],
  "monitorMode": true,
  "emitChangesOnly": true,
  "baselineOnly": false,
  "monitorKey": "nasa-news"
}

Keep monitorKey unchanged for runs that belong to the same monitor.

Advanced feed identity

For feeds whose URL changes, provide a stable feedId:

{
  "feeds": [
    {
      "url": "https://example.com/signed-or-changing-feed-url",
      "feedId": "company-news",
      "label": "Company news"
    }
  ],
  "monitorMode": true,
  "monitorKey": "competitive-intel"
}

The state record is keyed by the stable feed identity, not by batch membership. Adding another feed to the same batch does not reset existing feeds to NEW_ITEM.

Monitoring semantics

NEW_ITEM

The current item ID has not been observed before in this monitoring namespace.

Item identity is derived in this order:

1. RSS GUID or Atom ID
2. normalized item URL
3. title + published timestamp
4. content fingerprint fallback

GUID / Atom ID is preferred because content can change without changing item identity.

UPDATED_ITEM

A previously known item has the same stable item identity but one or more monitored fields changed.

The output includes changedFields, for example:

{
  "changeType": "UPDATED_ITEM",
  "changedFields": [
    "title",
    "summary",
    "content"
  ]
}

By default, a changing updatedAt timestamp alone does not trigger UPDATED_ITEM. Some feed generators refresh timestamps even when useful content is unchanged. Set compareUpdatedTimestamp to true if timestamp-only changes matter to your workflow.

UNCHANGED

The item is known and its monitoring hash is unchanged. With emitChangesOnly: true, these records are omitted from the dataset.

FEED_ERROR

The feed could not be downloaded or parsed. In changes-only mode, repeated identical feed failures are suppressed to avoid alert spam, while the run summary still reports the failure.

FEED_RECOVERED

A feed that failed on a previous monitoring run is healthy again. Previous item hashes are preserved across temporary failures, so recovery does not reset known items.

Conditional requests

When all of the following are true:

• monitorMode: true
• emitChangesOnly: true
• useConditionalRequests: true
• the previous response supplied ETag and/or Last-Modified

…the Actor sends If-None-Match and/or If-Modified-Since.

If the server returns HTTP 304 Not Modified, the Actor avoids reparsing the feed and emits no unchanged item records. Monitoring state and feed health are still updated.

When emitChangesOnly is false, the Actor fetches the full feed so it can return the current item snapshot.

Filters

Filtering affects emitted dataset records, but monitoring state still remembers parsed items. This prevents an item from being incorrectly classified as new just because a filter changed or the item started matching later.

Keyword filter

{
  "feedUrls": ["https://www.nasa.gov/feed/"],
  "includeKeywords": ["Artemis", "Moon"],
  "filterFields": ["title", "summary", "content", "categories"]
}

Include keywords use OR logic. If at least one matches, the item passes. Any exclude-keyword match suppresses the item.

Regex filter

{
  "feedUrls": ["https://example.com/feed.xml"],
  "includeRegex": ["\\bAI\\b", "machine\\s+learning"],
  "excludeRegex": ["sponsored"]
}

Regex matching is case-insensitive. Patterns are length/count limited and each search has a timeout to reduce ReDoS risk.

Pay-per-event pricing

The Actor is prepared for Apify pay-per-event monetization with two custom workload events:

• feed-checked — $0.00010 for each feed successfully fetched/checked, including HTTP 304 Not Modified monitoring checks.
• item-emitted — $0.00050 for each structured ITEM record emitted to the dataset.

FEED_ERROR and FEED_RECOVERED status records are not charged by item-emitted. Failed feeds are not charged by the custom workload events. A baselineOnly run or an unchanged 304 monitoring run therefore normally charges only the successful feed check and no item events.

For Store publication, the intended pricing setup is:

• synthetic apify-actor-start: $0.00005 per run;
• custom feed-checked: $0.00010;
• custom item-emitted: $0.00050;
• primary event: item-emitted;
• platform usage: included in the Actor price.

The runtime checks whether the current pricing model is pay-per-event before charging, so the same source remains usable during pre-publication/private testing. Spending-limit handling is state-safe: if the budget ends part-way through changed items, only paid/emitted changes advance item state; unpaid NEW_ITEM / UPDATED_ITEM changes remain eligible for a later run instead of being silently consumed.

Output

The default dataset contains two record types.

ITEM

Typical item:

{
  "recordType": "ITEM",
  "status": "SUCCESS",
  "feedId": "2fe1d11f...",
  "feedTitle": "Example News",
  "feedFormat": "RSS 2.0",
  "itemId": "9827b...",
  "itemKeySource": "GUID_OR_ATOM_ID",
  "sourceId": "item-001",
  "url": "https://example.com/article",
  "title": "Example headline",
  "author": "Example Author",
  "publishedAt": "2026-08-23T09:00:00Z",
  "updatedAt": null,
  "summary": "Short summary",
  "content": "Full content supplied by the feed",
  "text": "Example headline\n\nShort summary\n\nFull content supplied by the feed",
  "categories": ["Technology"],
  "enclosures": [],
  "contentHash": "...",
  "metadataHash": "...",
  "itemHash": "...",
  "changeType": "NEW_ITEM",
  "changedFields": [],
  "observedAt": "2026-08-23T10:00:00Z"
}

FEED_STATUS

Used for FEED_ERROR and FEED_RECOVERED events.

{
  "recordType": "FEED_STATUS",
  "status": "ERROR",
  "feedId": "...",
  "inputUrl": "https://example.com/feed.xml",
  "httpStatus": 503,
  "changeType": "FEED_ERROR",
  "errorType": "HTTP_ERROR",
  "errorMessage": "Feed request returned HTTP 503.",
  "observedAt": "2026-08-23T10:00:00Z"
}

Run summary

RUN_SUMMARY is stored in the default key-value store and includes:

• requested / successful / failed feeds
• HTTP 304 not-modified count
• parsed and matching item counts
• baseline-suppressed item count
• emitted item and feed-status counts
• monitoring change counts
• detected feed formats
• monitoring state writes
• per-feed processing errors

Feed formats and common fields

Format	Status
RSS 2.x	Supported
RSS 1.0 / RDF	Supported
Atom 1.0	Supported
Atom 0.3 common fields	Supported
JSON Feed	Not supported in v0.1.0
HTML feed auto-discovery	Not supported in v0.1.0

Common RSS namespaces are handled by local element names, including frequently used fields such as content:encoded and dc:creator.

If strict XML parsing fails, the Actor applies a conservative recovery pass for common public-feed defects such as HTML entities, bare ampersands, illegal XML 1.0 control characters, invalid numeric character references, and isolated encoding damage. Recovered feeds expose one of COMMON_XML_ENTITY_REPAIR, COMMON_XML_CHARACTER_REPAIR, or COMMON_XML_ENTITY_AND_CHARACTER_REPAIR in parseWarning, so downstream workflows can distinguish repaired XML from strictly valid XML.

Reliability and security

The Actor is designed for public HTTP(S) feeds.

Protections include:

• blocks localhost, loopback, private, link-local, reserved, multicast, and unspecified IP destinations
• resolves and checks the destination again on each redirect
• blocks redirect targets that become private-network URLs
• rejects non-HTTP(S) URLs
• rejects XML entity declarations and external DTD references while ignoring declaration-like text inside comments/CDATA
• retries strict-parse failures with a narrow repair pass for common HTML entities, unknown named entities, and bare ampersands
• bounded redirect count
• bounded retries for transient failures
• response-size limits applied while streaming decoded data
• regex timeouts
• persistent state limited per feed, with an effective cap never smaller than the current parsed item window
• temporary feed failures do not erase known item state

Limits

Defaults are intentionally conservative and configurable:

• up to 100 feeds per run by default, 1,000 maximum
• 500 items per feed by default, 5,000 maximum per feed
• 10 MB decoded response per feed by default, 50 MB maximum
• concurrency 10 by default, 30 maximum
• 2,000 remembered item IDs per feed by default, 10,000 maximum
• up to 250,000 output characters per text/summary/content field
• item URLs capped at 8,192 characters, categories/enclosures bounded, and large text fields capped to stay below Apify dataset item limits

The monitoring state stores hashes rather than complete historical article bodies, keeping persistent storage substantially smaller than saving full old feed content. If maxItemsPerFeed is configured above maxStateItemsPerFeed, the effective state cap is automatically raised to at least the parsed item window so current entries do not become false NEW_ITEM records on later runs.

Validation completed for v0.1.2

The source package includes automated tests and synthetic fixtures for RSS, Atom, RDF, monitoring transitions, HTTP behavior, security controls, state retention, filtering, malformed XML, batch behavior, pay-per-event charging, and spending-limit state safety.

Latest local validation:

• 173/173 automated tests passed under both unittest and pytest
• 95% statement coverage across src; src/main.py 100%, src/core.py 94%
• PPE tests cover successful feed/item charging, baselineOnly, unchanged 200, HTTP 304, free feed-error/recovery status behavior, failed-feed billing safety, feed-level charge limits, partial item limits, paid-update/unpaid-new state preservation, and 100-feed billing accounting
• 500 mutation-fuzz XML cases handled without unexpected exceptions
• additional 5,000 malformed-XML recovery cases parsed with 0 unexpected exceptions
• 5,000-item synthetic feed parsed successfully with ~20.24 MiB peak traced memory
• 5,000-item monitoring state serialized to ~2.53 MiB
• 100 mocked feeds / 10,000 total items processed successfully at roughly 10k items/s in the latest stress sanity run
• worst-case synthetic raw-HTML item test remained below 8 MB JSON, under Apify's 9 MB per-dataset-item ceiling

Apify Cloud functional validation also covered real RSS parsing, persistent baseline state, HTTP 304 Not Modified, FEED_ERROR, FEED_RECOVERED, UPDATED_ITEM, and NEW_ITEM. Controlled same-host benchmarks produced 100 items from 10 feeds in ~6 s / $0.001, 500 items from 50 feeds in ~11 s / $0.004, and a complete exported 1,000-item dataset from 100 feeds in ~25 s / $0.008. Those benchmark numbers are workload-specific and are not a guarantee for slower or larger third-party feeds.

Responsible use

Only process feeds you are allowed to access and reuse. Feed availability, content rights, robots/publisher policies, and downstream use remain the user's responsibility. This Actor does not bypass authentication, paywalls, or access controls.