# RSS Feed Scraper - Atom & JSON Feed Reader, New Item Monitor (`neverempty/rss-feed-scraper`) Actor

Give it feed URLs or plain website URLs. It finds the feed a site advertises, reads RSS 2.0, RSS 1.0/RDF, Atom and JSON Feed in any encoding, and returns one row per item with title, link, dates, author, summary, full text, categories, image and enclosures. Monitoring mode returns only new items.

- **URL**: https://apify.com/neverempty/rss-feed-scraper.md
- **Developed by:** [NeverEmpty](https://apify.com/neverempty) (community)
- **Categories:** News, Automation, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.38 / 1,000 feed item returneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## RSS Feed Scraper — read RSS, Atom and JSON Feed, find a site's feed, watch for new items

Give it feed URLs, or plain website URLs. `github.blog` is enough.

For a website, it reads the feed the page itself advertises with `<link rel="alternate" type="application/rss+xml">`
(or Atom / JSON Feed). Then it returns **one row per item**: title, link, published and updated dates, author,
summary, full text, categories, image, enclosures (podcast audio), duration, guid, and which feed it came from.

It reads **RSS 2.0, RSS 1.0 / RDF, Atom and JSON Feed (1.0 and 1.1)**, in any character encoding.

### Real feeds are messier than the specs

Measured on 2026-09-12 against more than 30 live feeds. Every case below is in this Actor's tests, using the real feed:

| Feed | What it does | What this Actor does |
|---|---|---|
| soumu.go.jp (Japanese government) | **Shift\_JIS**, and the server does not say so — only the XML declaration does | Reads the XML declaration, returns clean Japanese |
| Slashdot | Declares **ISO-8859-1** | Decoded as declared |
| kantei.go.jp (Japanese PM's office) | Served as `application/octet-stream`. guids are **positions** (`item_1` … `item_25`): tomorrow's `item_1` is a different article | Detects the feed from its content. Identifies an item by guid **and** link, so a new `item_1` is still new |
| abc.es | Lists **the same article 20 times**: one link for all of them, and the guid written both with and without a trailing slash | A trailing slash does not make a new item, so it is returned once and charged once. Items that really differ are still kept apart |
| A feed that stops mid-way | The server closes the connection, or a gzip body cannot be unpacked to the end | The item that was cut off is dropped, never returned and never charged. A free row says the body ended early |
| b.hatena.ne.jp | Every Japanese character in titles written as a hex entity (`&#x4E2D;`) | Decoded |
| gov.uk | Double-encoded summaries (`UK&amp;#39;s`) | `UK's` |
| blog.cloudflare.com, news.ycombinator.com | The page advertises the feed with a relative link (`/rss/`, `rss`) | Resolved against the page |
| YouTube channel pages | The feed link sits in `<body>`, not `<head>` | Found anyway |
| NPR podcast | 2.1 MB, 355 episodes in one feed | All read. Size limit is adjustable, and a feed cut at the limit returns only complete items |
| Most news feeds | Dates as `Fri, 11 Sep 2026 13:00:00 GMT`, `11 Sep 2026 19:00:00 +0900`, `2026-09-12T00:00:00+09:00` | Converted to UTC |

We also ran the same 16 feeds and 10 website URLs through a second, independent XML parser. Item counts,
links and titles matched on every feed we could compare. The check lives in `verify/crosscheck.mjs`, so you
can run it again.

### Dates are never guessed

- `publishedAt` / `updatedAt` hold a UTC instant **only when the feed states a time zone**.
- A time with no zone (`2026-09-11 10:00`) goes to `publishedLocal` / `updatedLocal` exactly as written. It is not
  treated as UTC, and it is not converted with the server's clock.
- Abbreviations that mean more than one zone (`IST`, `BST`) are treated as unknown.
- The raw text is always kept in `published` / `updated`.

### Monitoring mode: only new items

Turn on **Monitoring mode** and the Actor remembers the items it returned for each feed.

- The **first run** returns everything and remembers it.
- Later runs return **only items that were not returned before**.
- A quiet day returns a free row that says `no-new-since-last-run`. Nothing is charged. This is kept separate from
  `no-results`, which means the feed really lists nothing.
- Only items that were actually returned are remembered. If a run stops at a limit, the rest arrive on the next run.
- The memory is kept per feed and per filter setting, so two watches on the same feed with different keywords do
  not affect each other.
- The memory holds the **last 5,000 items** per feed and filter setting. A feed that publishes more than 5,000 items
  between two runs can return an older item again.

### What comes back

| Field | What it is |
|---|---|
| `title`, `link` | The item's title (plain text) and its absolute URL |
| `publishedAt`, `updatedAt` | UTC instant, only when the feed states a time zone |
| `publishedLocal`, `updatedLocal` | The time as written, when the feed gives no zone |
| `published`, `updated` | The raw date text from the feed |
| `author`, `authors` | Author names. **Email addresses are removed** (RSS writes authors as `email (Name)`) |
| `summary` | The description / summary as plain text |
| `contentText` | The full content as plain text, when the feed carries it (`content:encoded`, Atom `content`, JSON Feed `content_html` / `content_text`) |
| `contentHtml` | The same content as HTML, only with `includeContentHtml`. Relative links and images are made absolute |
| `hasFullContent` | Whether the feed carried full content for this item |
| `categories` | Categories, tags, `dc:subject` |
| `imageUrl` | `media:thumbnail`, an image `media:content` or enclosure, or `itunes:image`. Only images the feed itself names |
| `enclosureUrl`, `enclosureType`, `enclosureLength`, `enclosures` | Podcast audio and other attachments. A stated length of `0` means unknown and comes back as `null` |
| `durationSeconds` | From `itunes:duration` or JSON Feed attachments |
| `commentsUrl`, `externalUrl` | RSS `comments`, JSON Feed `external_url` |
| `guid`, `guidIsPermaLink`, `itemKey` | The feed's own id, and the key this Actor uses to recognise the item (guid + link) |
| `position`, `feedItemCount` | The item's place in the feed, and how many items the feed lists |
| `feedTitle`, `feedLink`, `feedLanguage`, `feedType`, `feedEncoding` | About the feed: `rss-2.0`, `rss-1.0-rdf`, `atom`, `json-feed-1.1`, …, and the encoding it was decoded with |
| `feedUrl`, `requestedUrl`, `foundVia` | The feed actually read (after redirects), the URL you gave, and `given` / `link-tag` / `common-path` |
| `status`, `reason`, `httpStatus` | `ok`, or why a URL produced nothing |

A field the feed does not state comes back as `null`. Nothing is filled in with a zero or a guessed date.

### Input

| Field | Default | What it does |
|---|---|---|
| `urls` | — | Feed URLs or website URLs. If this and `urlsText` are both empty, two example URLs are read and a free row says so |
| `urlsText` | empty | The same list as a block to paste, one URL per line |
| `maxItemsPerFeed` | 100 | Items per feed, in the feed's own order (newest first on almost every feed). 0 = all |
| `maxItems` | 1000 | Items across the whole run. URLs not reached get a free `not-looked-up` row. Must be at least 1: `0` is **not** "no limit" here, and a free row says so if you send it |
| `discoverAllFeeds` | off | From a website URL, read every advertised feed instead of the first one |
| `useCommonPaths` | on | When a page advertises no feed, try `/feed`, `/rss`, `/feed.xml`, `/rss.xml`, `/atom.xml`, `/index.xml`, `/feed.json` |
| `includeContentHtml` | off | Add `contentHtml` |
| `keywords`, `keywordMatch`, `excludeKeywords` | empty / any / empty | Match against title, summary, content, categories and author |
| `publishedAfter` | empty | `YYYY-MM-DD`. An item with no date is **kept**, because unknown is not the same as old |
| `timeoutSecs` | 20 | Per request |
| `maxFeedSizeMb` | 20 | Stop downloading a feed after this many MB. Only complete items are returned |
| `monitoringMode`, `resetMonitoringState` | off | See Monitoring mode |

### Pricing

Charged per item returned. The same item is never returned twice in one run, even when two feeds carry it or the
same feed is reached twice. Two feeds count as carrying the same article when the guid and link match (a trailing
slash is ignored) **or** when the title and publication time match. That second rule matters: Daring Fireball's Atom
feed and its JSON Feed carry the same 48 posts under completely different guids and links, so reading both with
`discoverAllFeeds` charges 48 items, not 96. A free row says how many items were folded that way.

Rows that explain why a URL produced nothing are **not charged**: bad address, blocked
(`blocked-to-us`, for example a Cloudflare "checking your browser" page), not a feed, no feed found, empty feed,
no match for the filters, nothing new in monitoring mode, an item already returned from another feed in the same
run, a body that ended in the middle, or not reached because of a limit.

### What it fetches

Only the URLs you give. For a website URL, it also fetches the feed that page advertises and, unless
`useCommonPaths` is off, up to 14 common feed locations: the seven names in the page's own folder and the same seven
at the site root. It does not follow links or crawl a site.

**robots.txt is not read or obeyed. This Actor fetches only the URLs you ask for**, the way a feed reader does
(Google documents the same approach for its Feedfetcher).

Addresses inside a private network are refused, before and after a redirect: `localhost`, `127.0.0.1`, `10.x`,
`172.16-31.x`, `192.168.x` and the cloud metadata address `169.254.169.254`. If you put a user name and password in a
URL (`https://user:pw@example.com/feed`), they are used to fetch the feed but are never written into `feedUrl`,
`requestedUrl` or the log.

### Notes

- Character encoding comes from the byte-order mark, then the document's own declaration, then the HTTP header, then
  UTF-8. When the declaration and the header disagree, the declaration wins. The exception is valid UTF-8 bytes
  where either side says UTF-8, because a common server default adds `charset=ISO-8859-1` to UTF-8 feeds.
- gzip is detected from the first two bytes of the file, not from the `.gz` in the name.
- Only `http` and `https` are accepted. `feed://` addresses are read as `http://`.
- Paged feeds (RFC 5005 `rel="next"`) are not followed. A run returns what the feed currently lists.

# Actor input Schema

## `urls` (type: `array`):

Feed URLs (RSS 2.0, RSS 1.0 / RDF, Atom or JSON Feed) or ordinary website URLs. For a website, the Actor reads the feed the page advertises with <link rel="alternate" type="application/rss+xml">. A bare domain such as github.blog is enough. If this and the paste box are both left empty, the two example URLs are read and a free row says so. If the two example URLs are still here when you give your own URLs in the paste box, they are skipped and not charged.

## `urlsText` (type: `string`):

The same list as a block you can paste, one URL per line. Both fields are merged and duplicates are dropped.

## `maxItemsPerFeed` (type: `integer`):

Return at most this many items from each feed, in the order the feed lists them (newest first on almost every feed). 0 = everything the feed lists.

## `maxItems` (type: `integer`):

Stop after this many items across all feeds in this run. URLs not reached are listed in free rows. This must be at least 1 and is not a way to say 'no limit': if 0 arrives from the API or a saved task, 1 is used and a free row says so. To read everything a feed lists, set Maximum items per feed to 0 and raise this number.

## `discoverAllFeeds` (type: `boolean`):

Off: from a website URL, read the first feed the page advertises (usually the main feed). On: read all of them, for example the posts feed and the comments feed. An article carried by two of those feeds is returned and charged only once: the same guid and link (a trailing slash is ignored), or the same title and publication time, counts as the same article, and a free row says how many were folded.

## `useCommonPaths` (type: `boolean`):

When a website URL advertises no feed, try /feed, /rss, /feed.xml, /rss.xml, /atom.xml, /index.xml and /feed.json in that folder and at the site root. Turn off to read only what a page officially declares.

## `includeContentHtml` (type: `boolean`):

Adds contentHtml: the item's full content exactly as the feed gives it (content:encoded, Atom content or JSON Feed content\_html), with relative links and images made absolute. The plain-text version, contentText, is always included.

## `keywords` (type: `array`):

Keep only items whose title, summary, content, categories or author contain these words (case does not matter). Empty keeps everything.

## `keywordMatch` (type: `string`):

any = contains at least one keyword. all = contains every keyword.

## `excludeKeywords` (type: `array`):

Drop items whose title, summary, content, categories or author contain any of these words.

## `publishedAfter` (type: `string`):

Keep items published (or, when the feed gives no publish date, updated) on or after this date, read as UTC. An item with no date at all is kept, because unknown is not the same as old.

## `timeoutSecs` (type: `integer`):

Give up on a single feed or page after this long. The other URLs are still read.

## `maxFeedSizeMb` (type: `integer`):

Stop downloading a feed after this many megabytes. The complete items read up to that point are returned and a free row says the feed was cut.

## `monitoringMode` (type: `boolean`):

Remembers the items it returned for each feed. The first run returns everything and remembers it. Later runs return only items that were not returned before. Nothing new means nothing returned and nothing charged.

## `resetMonitoringState` (type: `boolean`):

Clears the memory for the feeds in this input, so this run treats every item as new.

## Actor input object example

```json
{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml",
    "https://github.blog/"
  ],
  "urlsText": "",
  "maxItemsPerFeed": 100,
  "maxItems": 1000,
  "discoverAllFeeds": false,
  "useCommonPaths": true,
  "includeContentHtml": false,
  "keywords": [],
  "keywordMatch": "any",
  "excludeKeywords": [],
  "publishedAfter": "",
  "timeoutSecs": 20,
  "maxFeedSizeMb": 20,
  "monitoringMode": false,
  "resetMonitoringState": false
}
```

# Actor output Schema

## `results` (type: `string`):

Every item this run read out of the feeds, with its dates, author, summary, content, image and enclosures.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://feeds.bbci.co.uk/news/rss.xml",
        "https://github.blog/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("neverempty/rss-feed-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://feeds.bbci.co.uk/news/rss.xml",
        "https://github.blog/",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("neverempty/rss-feed-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml",
    "https://github.blog/"
  ]
}' |
apify call neverempty/rss-feed-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,neverempty/rss-feed-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/tF0rTYeGI6JuftUwE/builds/B6uQyLeywUCVxWNPW/openapi.json