# RSS Feed Reader with New-Item Detection (`madrasco/rss-feed-reader`) Actor

Read RSS 2.0 and JSON Feed into item rows, remember seen IDs in a named history store, and report feed status on scheduled repeats. Conditional GET and robots.txt checks are enabled by default.

- **URL**: https://apify.com/madrasco/rss-feed-reader.md
- **Developed by:** [Jack Valmadre](https://apify.com/madrasco) (community)
- **Categories:** News, Automation, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RSS Feed Reader with New-Item Detection

Read RSS 2.0 and JSON Feed into structured item rows for alerts and content pipelines, with a status row for each feed. Omit `stateStoreName` for a one-off read using the run's own storage. For recurring monitoring, supply a named history store and reuse it to output only previously unseen item IDs across runs.

Get titles, links, publication dates, authors, plain-text summaries, HTML content, categories and enclosures when supplied by the publisher. The actor reads feed content; it does not fetch linked articles or create AI summaries. It detects new IDs, so edits to an existing item do not trigger an update event.

### Start a watch list

Run with a feed URL and an explicit history-store name:

```json
{
  "feedUrls": ["https://www.jsonfeed.org/feed.json"],
  "stateStoreName": "my-feed-watch",
  "onlyNewItems": true,
  "maxItemsPerFeed": 100
}
```

The named store enables history between runs. If you omit `stateStoreName`, the actor uses the run's default key-value store and does not reuse history from previous runs. With that default, `onlyNewItems: true` still treats encountered IDs as new on each new run; the output-position limit still applies.

Reuse both the URL and store name on later runs. Store names use lowercase letters, digits and internal dashes, up to 63 characters; store IDs are not accepted. The supplied name requests read and write access to that store, including existing history.

With a persistent store, every encountered ID counts as new on the first successful read of a URL that has no saved history. For alerts about later posts, use that read as a baseline and start delivering alerts afterward. Configure scheduling and delivery separately in Apify or your automation. Do not overlap runs that use the same named store for the same feeds: history updates are not coordinated between runs.

### Use the output

The dataset contains three row types. Filter by `type` before using rows in an automation. Dataset views select columns; they do not filter out other row types.

| `type` | Contents |
|---|---|
| `item` | Feed content, `itemId`, `idSource`, `isNew`, `firstSeenAt`, `position` |
| `feed` | URL, `ok`, `status`, HTTP result, counts and errors |
| `summary` | Run totals, status counts, storage mode and URL-cap flag |

The summary reports `stateStore: null` and `historyPersistent: false` when the store name is omitted. With an explicit named store, `stateStore` contains that name and `historyPersistent` is `true`. These fields identify the selected storage mode; check each feed's status to confirm whether its history was saved successfully.

#### Example output

These selected fields come from a saved read of JSON Feed's public feed. The item is usable because its matching feed row has `ok: true`; the summary reports no failed feeds.

```json
[
  {
    "type": "item",
    "feedUrl": "https://www.jsonfeed.org/feed.json",
    "title": "Announcing JSON Feed",
    "isNew": true
  },
  {
    "type": "feed",
    "feedUrl": "https://www.jsonfeed.org/feed.json",
    "ok": true,
    "status": "ok"
  },
  {
    "type": "summary",
    "feedsFailed": 0
  }
]
```

To select usable items from a completed run's JSON dataset export, match each item's `feedUrl` to a successful feed row. This example reads the entire export before selecting items, because feed status can be written after item rows.

```python
import json

with open("dataset.json", encoding="utf-8") as source:
    rows = json.load(source)

successful_feeds = {
    row["feedUrl"] for row in rows
    if row.get("type") == "feed" and row.get("ok") is True
}
items = [
    row for row in rows
    if row.get("type") == "item"
    and row.get("feedUrl") in successful_feeds
]
```

The run's default key-value store also contains all feed status rows in `FEEDS` and the summary in `OUTPUT`. A successful actor run can contain failed feeds: check `feedsFailed` and the feed rows. Match item rows to a feed row with `ok: true` before delivering them. A history-write failure can leave item rows in the dataset without saving their IDs; those items can appear as new again later.

A successful repeat using persistent history can have no item rows. The actor can reuse saved HTTP validators for that feed, and HTTP 304 gives `status: "not-modified"` with no item rows even when `onlyNewItems: false`; it does not rebuild a full snapshot from cached content. Omitting `stateStoreName` starts without validators or seen IDs from previous runs.

### Options

| Field | Purpose | Default |
|---|---|---|
| `feedUrls` | Feed URLs or pages declaring a feed; aliases include `urls`, `startUrls`, `feeds`, `url` | Required |
| `stateStoreName` | Named store for history between runs; omit for run-local storage | Omitted |
| `onlyNewItems` | Emit only unseen IDs within the position limit; cross-run detection requires a named store | `false` |
| `maxItemsPerFeed` | First 1 to 10,000 feed positions eligible for output | `100` |
| `resetState` | Ignore earlier history for supplied feeds and replace it after successful reads | `false` |
| `includeContent` | Include `contentHtml`; plain-text summary remains when disabled | `true` |
| `discoverFeeds` | Follow the first feed link declared by an HTML page | `true` |
| `respectRobotsTxt` | Check robots.txt before feed requests and redirects | `true` |
| `maxConcurrency` | Process 1 to 20 feeds concurrently, subject to host pacing | `10` |
| `timeoutSecs` | Per-request timeout, from 5 to 120 seconds | `30` |

Supply a store name explicitly for recurring monitoring. Invalid supplied names are rejected. If the selected store cannot be opened, the run stops without switching histories or substituting run-local storage. Individual history read or write failures produce `internal-error` feed rows while other feeds continue. Omitting the name selects run-local storage from the start; it is not a fallback for named-store failures.

### Limits

- Up to 1,000 normalized, distinct URLs are accepted. Extra URLs are dropped and `feedCapReached` is set. This is an input cap, not a completion or throughput guarantee. Malformed URL strings are discarded without a feed row; no usable URL makes the run fail.
- `maxItemsPerFeed` limits positions before filtering for new IDs. IDs beyond that limit are still remembered and counted as new after a successful history write. With persistent history, raising the limit on a later run will not make those IDs new again.
- Identity uses the publisher's GUID or ID, then the item link, then a title-and-date hash. ID changes can repeat an article. In a named history store, a homepage URL and its direct feed URL maintain separate histories.
- History retains all IDs in the current feed plus earlier IDs within a target of 5,000 per URL. Larger current feeds can exceed that target. With persistent history, an old ID pruned from that history can count as new when it returns.
- Same-host requests run serially with a one-second gap after each response. Different hostnames and separate runs do not share pacing. Repeated HTTP 429, 503 or network failures trigger backoff and a stop for that host within the process.
- Responses are capped at 15 MB after decompression, with at most five redirects. Truncated XML may retain completed items with `ok-with-errors`; inspect parse errors. Nonstandard dates may be `null`.
- HTML content is publisher-supplied and is not sanitized for display. Login pages, paywalls and bot checks are not bypassed. Discovery follows a declared feed link; it does not search every possible feed location.
- Real-source release validation covers RSS 2.0 and JSON Feed. Atom, RDF and older RSS parser paths have bundled-example checks, without retained real-source validation for this release.

Check publisher terms and your right to reuse content. Robots.txt checking does not grant permission or guarantee compliance. Disable it only for feeds you own or have permission to read.

### Cost

No developer rental or per-result fee. Apify platform usage and storage charges may still apply under your account's plan. There is no measured cost-per-1,000-feeds estimate for this release.

### Privacy and support

Feed content, publisher-supplied author names, feed status and summaries are saved in your Apify account. History records contain feed URLs, HTTP validators, encountered IDs and first-seen times. With an explicit `stateStoreName`, these records are reused from the named store between runs. With the name omitted, they are stored in the run's default key-value store and are not reused automatically by later runs. Run-local storage does not mean automatic deletion: the actor does not delete outputs or expire history by age. Your Apify retention and sharing settings govern stored data. Requests disclose requested URLs to publisher and redirect servers; avoid URLs containing private tokens. `resetState` does not delete previous run outputs or unrelated history.

Built and maintained by AI agents for Madrasco. Use the actor's Issues tab or support@madrasco.dev for support. Platform replies may be AI-assisted; a human owner can be reached on request.

# Actor input Schema

## `feedUrls` (type: `array`):

RSS 2.0 or JSON Feed URLs. Up to 1,000 normalized unique URLs are accepted; excess URLs are ignored and feedCapReached is set in the summary. This input cap is not a guarantee of completion within your run timeout. Same-host requests run serially with a one-second gap after each response. A website or blog page can declare its feed with a <link rel="alternate"> tag. Also accepted as urls, startUrls, feeds or a single url. Other parser formats need further real-source release verification.

## `onlyNewItems` (type: `boolean`):

Leave out IDs already seen in the selected history, subject to maxItemsPerFeed's position limit. Cross-run detection requires an explicit stateStoreName reused on later runs. If that field is omitted, each new run starts without earlier history, so encountered IDs count as new again. Feed status rows are still written.

## `maxItemsPerFeed` (type: `integer`):

Only the first 1 to 10,000 positions in each feed are eligible for output, before filtering for new IDs. IDs beyond that limit are still remembered and counted as new after a successful history write. With persistent history, raising the limit on a later run will not make those IDs new again.

## `stateStoreName` (type: `string`):

Supply a named key-value store in your Apify account to remember seen IDs and HTTP validators between runs. Omit this field for a one-off read: history is stored in the run's default key-value store and is not reused by later runs. The summary then has stateStore: null and historyPersistent: false; an explicit named store reports its name and historyPersistent: true. A supplied name requests read/write access to that store, including existing history. Use lowercase letters, digits and internal dashes, up to 63 characters; store IDs are not supported. Invalid names are rejected, and failure to open a selected store stops the run without fallback. Per-feed history read/write failures produce internal-error rows while other feeds continue. Check feed status before using item rows: a history write can fail after items were emitted. Avoid overlapping runs for the same feeds and named store.

## `resetState` (type: `boolean`):

Ignore earlier history for the supplied feeds and send no conditional request based on it. Every encountered ID counts as new; the output-position limit still applies. Replace those feeds' history after successful reads. With stateStoreName omitted, each new run already starts without earlier history. This option does not delete previous run outputs or unrelated feed histories.

## `includeContent` (type: `boolean`):

Keep each item's full HTML (content:encoded, Atom content, content_html) in 'contentHtml'. Turn off for smaller rows; the plain-text 'summary' is always included.

## `discoverFeeds` (type: `boolean`):

If a URL returns an HTML page, follow the first feed link the page declares (one extra request).

## `respectRobotsTxt` (type: `boolean`):

Skip feeds that the site's robots.txt disallows, including redirect targets, (reported as 'blocked-by-robots'). Turn off only for feeds you own or have the publisher's permission to read.

## `maxConcurrency` (type: `integer`):

How many feeds are fetched in parallel.

## `timeoutSecs` (type: `integer`):

Per request.

## Actor input object example

```json
{
  "feedUrls": [
    "https://www.jsonfeed.org/feed.json"
  ],
  "onlyNewItems": false,
  "maxItemsPerFeed": 100,
  "resetState": false,
  "includeContent": true,
  "discoverFeeds": true,
  "respectRobotsTxt": true,
  "maxConcurrency": 10,
  "timeoutSecs": 30
}
```

# Actor output Schema

## `items` (type: `string`):

No description

## `feeds` (type: `string`):

No description

## `feedStatus` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://www.jsonfeed.org/feed.json"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("madrasco/rss-feed-reader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "feedUrls": ["https://www.jsonfeed.org/feed.json"] }

# Run the Actor and wait for it to finish
run = client.actor("madrasco/rss-feed-reader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://www.jsonfeed.org/feed.json"
  ]
}' |
apify call madrasco/rss-feed-reader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,madrasco/rss-feed-reader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/4WaqjbcTV3H461zDE/builds/CSJQcvtekvCp2t5qR/openapi.json