# Website to RSS / Change Monitor (`rationalistic_tail/website-to-rss-monitor`) Actor

Turn public websites into RSS, Atom, and JSON Feed. Auto-detects articles, obeys robots.txt, optional Playwright, and monitors for new items across runs with webhooks. Pay $0.015 per new item. No login or paywall bypass.

- **URL**: https://apify.com/rationalistic_tail/website-to-rss-monitor.md
- **Developed by:** [Jerry Lin](https://apify.com/rationalistic_tail) (community)
- **Categories:** Developer tools, Automation, News
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $15.00 / 1,000 new feed items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website to RSS / Change Monitor

Turns **public** website listing pages into **RSS 2.0**, **Atom 1.0**, and **JSON Feed 1.1**, with optional **stateful change detection** and webhooks.

> Always obeys `robots.txt`. No login / paywall bypass. Identifying User-Agent. Polite rate limits.

### What it does

| Capability | Details |
|---|---|
| Multi-URL input | Process a list of public index / listing pages in one run |
| Auto-detect | Heuristics: `<article>` cards, heading links, repeated link blocks, dates |
| Manual selectors | Optional CSS: `item`, `title`, `link`, `date`, `summary` |
| Fetch modes | Default: plain HTTP + Cheerio. Optional: Playwright for JS-rendered pages |
| Feeds | RSS 2.0, Atom, JSON Feed written to the run key-value store |
| Change monitor | Named KV store keyed by source URL; emit / charge **only new items** |
| Webhook | Optional POST when new items appear |
| robots.txt | Always enforced. Disallowed URLs → `blocked_by_robots`, **never charged** |

### Inputs (summary)

| Field | Default | Notes |
|---|---|---|
| `urls` | required | Public `http(s)` listing URLs |
| `maxItemsPerUrl` | `30` | Cap per source |
| `outputFormats` | `rss, atom, json` | KV feed files |
| `usePlaywright` | `false` | Enable for JS-heavy sites |
| `itemSelector` etc. | empty | Manual CSS overrides |
| `stateStoreName` | `website-rss-monitor-state` | Cross-run memory |
| `emitOnlyNew` | `true` | Dataset + PPE only for new items |
| `webhookUrl` | empty | JSON POST on new items |
| `requestDelayMs` | `1000` | Polite delay |
| `maxConcurrency` | `1` | Keep low |

### Output

#### Dataset items

```json
{
  "sourceUrl": "https://www.example.gov/news",
  "url": "https://www.example.gov/news/123",
  "title": "Announcement title",
  "summary": "…",
  "date": "2026-11-01T00:00:00.000Z",
  "isNew": true,
  "status": "ok",
  "detectedAt": "2026-11-09T13:00:00.000Z"
}
```

`status` may also be `blocked_by_robots`, `fetch_error`, or `no_items`.

#### Key-value store

- `feed_<host>_<hash>.xml` — RSS 2.0
- `feed_<host>_<hash>.atom` — Atom 1.0
- `feed_<host>_<hash>.json` — JSON Feed 1.1
- `OUTPUT_SUMMARY` — run summary JSON

On the Apify platform, KV records get public URLs you can subscribe to from feed readers (re-run on a schedule to refresh).

### How much does it cost?

Pay-per-event pricing – no subscription, no separate compute bill:

| Event | Price |
|---|---|
| `feed-item` – one **new** feed item in the dataset | **$0.015** |
| Actor start (per GB of memory; default 1 GB = 1 event) | **$0.005** |

**Never charged as `feed-item`:** `blocked_by_robots` rows, fetch errors, empty extracts, or previously seen items (when `emitOnlyNew` is true).

#### Example run costs

| Scenario | Events | Approx. cost |
|---|---|---|
| Monitor tick, 0 new posts | start only | **$0.005** |
| First snapshot, 1 new item | start + 1 | **$0.02** |
| First snapshot, 10 new items | start + 10 | **$0.155** |
| Daily delta, 3 new items | start + 3 | **$0.05** |

Formula: `0.005 + 0.015 × N` new items (at the default 1 GB memory).

### Legal / responsibility

- This Actor only fetches **publicly reachable** pages you supply.
- It **checks and obeys `robots.txt`**. Disallowed URLs are skipped and never billed.
- It does **not** log in, solve CAPTCHAs, or bypass paywalls / access controls.
- **You (the run operator)** are responsible for ensuring your use of extracted public content complies with the target site’s terms, copyright, and applicable law.
- Prefer official feeds / open data APIs when a site already provides them.

### Use with AI agents (MCP)

Add this Actor as a tool in any MCP client (Claude, Cursor, VS Code, ChatGPT connectors) through the official Apify MCP server:

```
https://mcp.apify.com?tools=rationalistic_tail/website-to-rss-monitor
```

Example agent request: *"What's new on blog.cloudflare.com since last time?"* → the agent calls the tool with `{"urls": ["https://blog.cloudflare.com/"]}` and reads the new items from the dataset (or `OUTPUT_SUMMARY`). Keep `stateStoreName` the same between runs to get only new items. The Actor runs with **limited permissions**, so it also works with prepaid agent tokens from Apify AGI (x402 / Skyfire).

***

### 繁體中文（簡短）

**網站轉 RSS / 更新監控** — 把公開網頁列表轉成 RSS、Atom、JSON Feed；可自動偵測文章區塊，也可自訂 CSS；預設用 HTTP+Cheerio，需要時才開 Playwright。會**遵守 robots.txt**（禁止則標記 `blocked_by_robots` 且**不收費**），不做登入或付費牆繞過。跨 run 用命名 KV store 做變更偵測，只對**新項目**計費。

計費採 pay-per-event：每筆新項目 `feed-item` **USD 0.015**，每次執行起始費 **USD 0.005**；被 robots.txt 擋下、抓取失敗或已看過的項目不收費。

**法律／責任聲明：** 僅抓取你提供的公開網址；請自行確認目標網站條款與著作權是否允許此用途。官方已有 RSS／開放資料時請優先使用官方來源。

# Actor input Schema

## `urls` (type: `array`):

Public listing / index page URLs to convert into feeds, e.g. \["https://blog.cloudflare.com/"]. Only publicly reachable pages; this Actor never logs in or bypasses paywalls.

## `feedTitle` (type: `string`):

Optional feed title. Default: page <title> or hostname.

## `feedDescription` (type: `string`):

Optional feed description.

## `maxItemsPerUrl` (type: `integer`):

Maximum items extracted from each source URL (after detection).

## `outputFormats` (type: `array`):

Feed formats to write into the key-value store.

## `usePlaywright` (type: `boolean`):

If true, render pages with Playwright/Chromium. Default false uses plain HTTP + Cheerio (cheaper, faster).

## `itemSelector` (type: `string`):

Manual override: CSS selector for each list item / article card. Leave empty for auto-detect.

## `titleSelector` (type: `string`):

Relative to item (or absolute). Empty = auto.

## `linkSelector` (type: `string`):

Relative to item. Empty = first meaningful <a href>.

## `dateSelector` (type: `string`):

Relative to item. Empty = look for <time> / date-like text.

## `summarySelector` (type: `string`):

Relative to item for description/summary text.

## `stateStoreName` (type: `string`):

Named key-value store used for cross-run change detection (keyed by source URL). Empty = use default store (local only / not shared across runs on platform unless named).

## `emitOnlyNew` (type: `boolean`):

If true, dataset push + PPE charge only for items not seen in previous runs. Feed files still include the latest extracted items.

## `webhookUrl` (type: `string`):

Optional URL to POST when new items are found. Body is JSON with sourceUrl + items. Empty = no webhook.

## `requestDelayMs` (type: `integer`):

Minimum delay between outbound HTTP/browser requests to the same host.

## `maxConcurrency` (type: `integer`):

Max parallel page fetches. Keep low to stay polite.

## `userAgent` (type: `string`):

Optional custom User-Agent. Default identifies this Actor clearly.

## Actor input object example

```json
{
  "urls": [
    "https://blog.cloudflare.com/"
  ],
  "feedTitle": "",
  "feedDescription": "",
  "maxItemsPerUrl": 30,
  "outputFormats": [
    "rss",
    "atom",
    "json"
  ],
  "usePlaywright": false,
  "itemSelector": "",
  "titleSelector": "",
  "linkSelector": "",
  "dateSelector": "",
  "summarySelector": "",
  "stateStoreName": "website-rss-monitor-state",
  "emitOnlyNew": true,
  "webhookUrl": "",
  "requestDelayMs": 1000,
  "maxConcurrency": 1,
  "userAgent": ""
}
```

# Actor output Schema

## `dataset` (type: `string`):

Feed items and status markers pushed to the default dataset.

## `outputSummary` (type: `string`):

JSON run summary with per-URL statuses, counts, and generated feed keys.

## `rssFeeds` (type: `string`):

RSS 2.0 feed bodies stored under FEED_RSS\_<host>\_<pathHash> keys. Browse keys then open /records/<KEY>.

## `atomFeeds` (type: `string`):

Atom 1.0 feed bodies stored under FEED_ATOM\_<host>\_<pathHash> keys. Browse keys then open /records/<KEY>.

## `jsonFeeds` (type: `string`):

JSON Feed 1.1 bodies stored under FEED_JSON\_<host>\_<pathHash> keys. Browse keys then open /records/<KEY>.

## `exampleRssRecord` (type: `string`):

Replace FEED_RSS_example.com_abc123 with a FEED_RSS\_\* key from OUTPUT_SUMMARY.results\[].feedKeys.rss.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://blog.cloudflare.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rationalistic_tail/website-to-rss-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://blog.cloudflare.com/"] }

# Run the Actor and wait for it to finish
run = client.actor("rationalistic_tail/website-to-rss-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://blog.cloudflare.com/"
  ]
}' |
apify call rationalistic_tail/website-to-rss-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rationalistic_tail/website-to-rss-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/tmLS8Ps70fUj1TSLK/builds/zHhVu3uKheiFHsXcP/openapi.json
