RSS & Atom Feed Change Detector

Give it a list of feed URLs (RSS 2.0, Atom or RDF). Each run fetches every feed once and outputs only the items that were not there in earlier runs. Every item comes out in the same shape, whatever the feed format: feed URL, feed title, title, link, published date (ISO 8601), author, plain-text summary and guid.

Schedule it every 15 minutes or every hour and connect a webhook, Slack or email, and you get the new posts of the sites you follow without reading the feeds yourself. Keyword filters let you keep only the items you care about.

Each feed gets one GET request. The request is conditional: the Actor sends back the ETag and Last-Modified the server gave on the previous run, so a feed that did not change answers 304 Not Modified and nothing is downloaded. Requests identify themselves with the User-Agent Mozilla/5.0 (compatible; ApifyFeedChangeDetector/1.0; +https://apify.com/isnard/rss-feed-change-detector).

Input

{
    "feedUrls": ["https://hnrss.org/frontpage", "https://blog.python.org/feeds/posts/default"],
    "includeKeywords": ["python", "security"],
    "excludeKeywords": ["sponsored"],
    "maxItemsPerFeed": 100,
    "onlyNew": true,
    "stateKey": ""
}

• feedUrls (required): feed URLs, one per line. Duplicates are fetched once. Lines that are not http(s) URLs are skipped with a warning.
• includeKeywords (optional): keep only items whose title or summary contains at least one of these words. Case-insensitive, plain substring match.
• excludeKeywords (optional): drop items whose title or summary contains any of these words.
• maxItemsPerFeed (default 100): only the first N items of each feed are looked at. Most feeds list the newest first.
• onlyNew (default true): output only items not seen before. When off, every item that passes the filters is output, with isNew set to true or false, and the feed is always downloaded in full.
• stateKey (optional): name for the memory of seen items. See below.

How "new" is decided

Every item gets an id: a hash of its guid (RSS), id (Atom) or rdf:about (RDF). When the feed has none, the link is used, and when there is no link either, the title plus the date. An item is new when its id is not in the list stored by previous runs.

The lists live in a named key-value store called rss-feed-change-detector-state in your Apify account, one record per feed, with the last 2,000 ids, the ETag and the Last-Modified value. Items removed by the keyword filters are remembered too, so they are not checked again.

The record name is made from the stateKey and the feed URL. Without a stateKey, every run that watches the same feed with the same keyword filters shares the record. That means adding a feed to a task does not reset the others, and two tasks with different filters do not hide items from each other. Set a stateKey when you want a task to keep its own memory, for example two tasks that watch the same feed for different people. To start fresh, use a new stateKey or delete the store in Storage > Key-value stores.

The first run for a feed has nothing to compare with, so every item it reads (up to maxItemsPerFeed) is new and charged. Lower maxItemsPerFeed if you do not want that backlog. If you raise it later, older items that were never read come out as new.

Output

One dataset item per new item. Real example from a test run (more in ejemplos/salida.json in the source):

{
    "feedUrl": "https://blog.python.org/feeds/posts/default",
    "feedTitle": "Python Insider",
    "title": "Python 3.15.2, 3.14.10, 3.13.18, 3.12.16 and 3.11.18 are out",
    "link": "https://blog.python.org/2026/10/python-3152-31410-31318-31216-31118/",
    "published": "2026-10-26T00:00:00+00:00",
    "author": "Hugo van Kemenade",
    "summary": "Five expedited security releases to distrust the compromised eMudhra intermediate CA keys.",
    "guid": "https://blog.python.org/2026/10/python-3152-31410-31318-31216-31118/",
    "isNew": true,
    "checkedAt": "2026-10-29T09:36:17+00:00"
}

• published: pubDate, dc:date, published or updated, converted to ISO 8601. Dates without a time zone are taken as UTC. null when the feed gives no date or one that cannot be read.
• summary: description, summary or content, with HTML removed, whitespace collapsed and cut at 1,000 characters.
• link: relative links are resolved against the feed URL. For Atom, the alternate link is used.

A feed that cannot be read (HTTP error, timeout, DNS error, not a feed, broken XML) gets one item with feedUrl, checkedAt and error, for example "error": "Got an HTML page, not a feed". The run goes on with the other feeds. Feeds with common mistakes, like a bare &, HTML entities such as   or stray control characters, are repaired before parsing.

The run's status message gives the totals, for example 12 new items from 5 feeds. 3 not modified, 0 failed.

Scheduling

1. Open the Actor in Apify Console, fill in the input and click Save as a new task.
2. Go to Schedules, click Create new and set a cron expression, for example */15 * * * * (every 15 minutes) or 0 * * * * (every hour).
3. Add the task to the schedule and save.

Each scheduled run compares with the previous one through the state store. Do not run two copies of the same task at the same time: both would see the same items as new.

Getting the new items somewhere

On the task, open the Integrations tab.

• Slack: add the Slack integration on "Run succeeded" and pick a channel. The message includes the status message with the number of new items.

• Email: add the email integration on "Run succeeded". Works best with hourly or daily schedules.

• Webhook: add an HTTP webhook on "Run succeeded" pointing to your endpoint, or to Make, Zapier or n8n. The payload has resource.defaultDatasetId. Fetch the items with:

  GET https://api.apify.com/v2/datasets/<defaultDatasetId>/items?clean=true&fields=title,link,feedTitle,published
  

  Skip items that have an error field, and do nothing when the list is empty.

A minimal webhook receiver that posts each new item to a Slack incoming webhook (Python, standard library only):

import json, urllib.request

def handle(payload: dict, apify_token: str, slack_url: str) -> None:
    dataset = payload['resource']['defaultDatasetId']
    url = f'https://api.apify.com/v2/datasets/{dataset}/items?clean=true&token={apify_token}'
    items = json.load(urllib.request.urlopen(url))
    for item in items:
        if item.get('error'):
            continue
        text = f"{item['feedTitle']}: <{item['link']}|{item['title']}>"
        body = json.dumps({'text': text}).encode()
        urllib.request.urlopen(urllib.request.Request(slack_url, body, {'Content-Type': 'application/json'}))

Pricing

Pay per event:

• new-item: each new item written to the dataset.

Feeds that fail, feeds that did not change and items already seen are not charged. With onlyNew off, items that were already seen are output without charge. You can cap the cost of a run with the "Maximum cost per run" setting. When the cap is reached the Actor stops, and the items it did not output stay unseen, so the next run picks them up.

Limitations

• Only RSS 2.0, Atom and RDF (RSS 1.0). JSON Feed is not supported.
• XML that is broken beyond the simple repairs (for example cut off in the middle) is reported as an error.
• No login, cookies or custom headers. Feeds behind authentication cannot be read.
• Keyword filters are substring matches: art also matches party.
• Only the first maxItemsPerFeed items are read. If a feed publishes more than that between two runs, the extra items are missed. Schedule more often or raise the limit.
• Only the last 2,000 ids per feed are remembered. A feed that brings back very old items can make them show as new.
• If a publisher changes the guid of an existing item, it shows up as new.
• Some sites block automated requests and answer 403. That is reported as an error item.

Run locally

python -m venv .venv
.venv/Scripts/pip install -r requirements.txt   # on Linux/macOS: .venv/bin/pip
# put input in storage/key_value_stores/default/INPUT.json
.venv/Scripts/python -m my_actor
python -m unittest discover -s tests -t .

Author: Matias Isnard.