RSS and Atom Change Monitor

Get new or updated headlines from public feeds without sending the same items through your workflow again.

• Combine up to 10 HTTPS RSS 2.0 or Atom 1.0 feeds.
• Filter headline/source keywords and optional age; export structured metadata.
• Remember previous runs and report new, updated, or optionally unchanged items.

Useful for monitoring publication feeds and feeding a research queue, dashboard, or automation. This tool does not write articles, make decisions, send messages, or find customers. No LLM key, browser, or third-party Python package is required.

Quick input

{
  "feedUrls": ["https://feeds.bbci.co.uk/news/technology/rss.xml"],
  "newOnly": true,
  "maxItems": 200,
  "monitorName": "technology-news"
}

First run returns existing matching items. Later sequential runs return new or changed metadata. baselineOnly: true suppresses the first run's items while seeding history. Change monitorName to start fresh. Use the same input configuration to continue a monitor. Changing feed URLs, keywords or age filter creates a separate history.

An output row contains id, title, url, publishedAt, source, feedUrl, change, and observedAt. See sample-output.json for three real rows from the verification run. Feed content and links belong to their publishers; no full articles are copied.

Local tryout

Requires Python 3.12 or newer. On Windows double-click TRY_LOCAL.cmd. Or run:

python feed_monitor.py --input INPUT.json

Results: local-output/items.json; diagnostics: local-output/summary.json; history: local-output/state.json. Delete the local state file to reset. This runs on demand and does not schedule anything or earn money locally.

Behavior and limits

HTTPS feeds only, port 443, UTF-8 encoding, no login/cookies. Up to 2 MB per feed and 500 returned items per run. No archive scraping: results are limited to the entries the feed actually exposes. This does not promise every story from a publisher.

Matching uses case-insensitive substrings in headline/source, not semantic AI classification. Deduplication uses article links with common tracking parameters removed; entries without links use feed-scoped IDs. Different URLs for the same underlying story may remain separate. Updates mean changes in the supplied headline/link/date/source, not changes to the full article body.

Unknown dates are null. Age filtering skips and counts them. Missing entries are not marked deleted: RSS feeds are rolling windows. Partial failures are recorded in SUMMARY on Apify. If all feeds fail, the run fails and history is not overwritten. A successful no-change run returns an empty dataset.

History retains at most 5,000 identities per monitor, so very old evicted items can appear again. State is committed after output; an interrupted run can repeat results. Delivery is not exactly-once. Use id downstream for idempotency. Do not overlap runs of the same monitor: cloud storage has no compare-and-swap lock in this pilot.

For ordinary cloud networking, connections resolve and validate public addresses before connecting. If an environment requires an HTTPS proxy, the shipped proxy route accepts only feeds.bbci.co.uk, including redirects. Other feeds require direct public DNS/network access. This restriction cannot be changed through user input.

Apify package status

apify_main.py reads platform input, writes the default dataset and SUMMARY, and stores history in an actor-created named key-value store. It uses the invoking account's token from the environment, never a publisher's hardcoded token. Keep Limited permissions. Creating a named store under those permissions is documented to retain access on subsequent runs by the same user: Apify storage permissions.

Live cloud tests verified RSS and Atom parsing, persistent history, duplicate suppression across sequential runs, partial-feed failure handling, and output limits. Billing and payouts have not been fully verified.

Pricing

$0.001 per Actor start plus $0.00002 per returned item ($0.02 per 1,000 items). A no-result run costs $0.001; a 100-item run costs $0.003. Run platform usage is included. Post-run storage and access may incur standard Apify charges. See the Actor's Pricing tab for current rates.

Tests

python -m unittest discover -s tests -v

21 checks plus two live sequential feed runs are documented in ../VERIFICATION.md.