# RSS & Atom Feed Scraper: Universal Content Monitor (`arman-bd/rss-feed-scraper`) Actor

Parse any RSS, Atom or JSON feed into a single normalised schema: title, link, author, published date, categories, media enclosures and full content. Auto-discovers the feed from a plain site URL.

- **URL**: https://apify.com/arman-bd/rss-feed-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** News, Social media, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 item scrapeds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## RSS & Atom Feed Scraper: Universal Content Monitor

![RSS Feed Scraper: One normalised record per feed item, title, link, author, published date, categories and full article body](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/rss-feed-scraper.jpg)

**RSS & Atom Feed Scraper** reads any RSS, Atom or JSON feed on the web and returns one flat, identically-shaped record per item, title, link, author, published date, categories, media enclosures and the full article body.

Syndication feeds are the oldest public API on the internet: publishers maintain them on purpose, for machines. This Actor reads them directly. **No proxy setup, no browser, no credentials to manage.** Point it at a feed URL, or at a plain site homepage, the page's advertised feed is discovered and followed for you.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/rss-feed-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/rss-feed-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `requestedUrl`, `feedUrl` | What you asked for, and the feed that was actually read after discovery and redirects |
| `feedTitle`, `feedType` | Channel title, and which format it turned out to be, `rss`, `rss-1.0`, `atom` or `json` |
| `itemId` | The feed's own identity for the item, RSS `guid`, Atom `id`, JSON Feed `id` |
| `title`, `link` | Headline and permalink, resolved to an absolute URL |
| `author` | `dc:creator`, Atom `<author><name>` or JSON Feed `authors[]`, whichever the feed uses |
| `publishedAt`, `updatedAt` | Always ISO-8601, whether the feed wrote RFC-822 or ISO-8601 |
| `summary` | Short description as plain text |
| `contentText` | Full article body as plain text, HTML stripped, entities decoded |
| `contentHtml` | The publisher's original markup, when you ask to keep it |
| `categories` | Tags and categories the item is filed under |
| `enclosures` | Attached media as `{ url, type, length }`, podcast audio, video, PDFs |
| `imageUrl` | Lead image from `media:content`, `media:thumbnail`, `itunes:image`, an image enclosure, or the first `<img>` in the body |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, a per-feed breakdown, the filters used, and any source that failed.

### Common use cases

- **Competitor and newsroom monitoring.** Watch dozens of blogs on a schedule and diff on `itemId`.
- **News aggregation.** One normalised schema across hundreds of publishers, so downstream code has one shape to handle.
- **Content curation pipelines.** Filter by keyword at ingest and hand the rest to a summariser.
- **Brand monitoring.** Track mentions across the feeds of trade press and forums.
- **Archiving.** Snapshot full article bodies before publishers rewrite or unpublish them.

### Quick start

Three feeds, three different formats, everything they carry:

```json
{
 "feedUrls": [
 "https://news.ycombinator.com/rss",
 "https://blog.rust-lang.org/feed.xml",
 "https://daringfireball.net/feeds/json"
 ]
}
```

Homepages instead of feeds, the Actor finds them:

```json
{
 "feedUrls": ["https://arstechnica.com", "github.blog", "https://blog.rust-lang.org/"],
 "maxItemsPerFeed": 20
}
```

A filtered, recent, headline-only sweep for change detection:

```json
{
 "feedUrls": ["https://feeds.bbci.co.uk/news/rss.xml"],
 "sinceDate": "2026-08-01",
 "keywordFilter": ["election", "budget"],
 "includeContent": false
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `feedUrls` | array | - | **Required.** Feed URLs or site homepages. A bare host (`github.blog`) is accepted and gets `https://` prepended. |
| `maxItemsPerFeed` | integer | `0` | Cap saved items per feed, applied **after** filtering. `0` = no limit. |
| `sinceDate` | string | `""` | Keep only items published after this date. `2026-01-01` or a full ISO-8601 timestamp. |
| `keywordFilter` | array | `[]` | Keep only items whose title, summary or categories contain one of these terms. Empty = all. |
| `includeContent` | boolean | `true` | Include the full body in `contentText`. Off gives a much smaller dataset. |
| `stripHtmlTags` | boolean | `true` | On, bodies are plain text only. Off additionally keeps the original markup in `contentHtml`. |

Filters combine with AND: an item must pass `sinceDate` *and* match at least one keyword. Items with no publish date at all are never dropped by `sinceDate`.

### Output example

```json
{
 "requestedUrl": "https://arstechnica.com/",
 "feedUrl": "https://arstechnica.com/feed/",
 "feedTitle": "Ars Technica",
 "feedType": "rss",
 "itemId": "https://arstechnica.com/gadgets/2026/08/ai-isnt-enough-to-protect-social-media-communities-from-ai/",
 "title": "AI isn't enough to protect social media communities from AI",
 "link": "https://arstechnica.com/gadgets/2026/08/ai-isnt-enough-to-protect-social-media-communities-from-ai/",
 "author": "Scharon Harding",
 "publishedAt": "2026-08-06T11:00:44.000Z",
 "updatedAt": null,
 "summary": "Why humans need to moderate humans.",
 "contentText": "Sometimes you have to fight fire with fire. But when it comes to AI slop and hateful content threatening the safety and value of social media platforms …",
 "contentHtml": null,
 "categories": ["AI", "Tech", "discord", "Facebook", "generative ai", "reddit", "social media"],
 "enclosures": [],
 "imageUrl": "https://cdn.arstechnica.net/wp-content/uploads/2026/07/GettyImages-2259634870-1152x648.jpg",
 "scrapedAt": "2026-08-06T11:43:41.586Z"
}
```

### Finding a feed URL

You usually do not have to. Paste the site and let the Actor look:

1. The URL is fetched and parsed as a feed. If it is one, done.
2. If it is an HTML page, its `<link rel="alternate" type="application/rss+xml">` tags are read and the first advertised feed is followed. Atom and JSON Feed `type` values work the same way.
3. If the page advertises nothing, the conventional paths are tried in order: `/feed`, `/rss`, `/rss.xml`, `/feed.xml`, `/atom.xml`, `/index.xml`.

If you would rather be explicit, most publishers link their feed in the page footer, and WordPress sites are almost always at `/feed/`.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~rss-feed-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "feedUrls": ["https://news.ycombinator.com/rss", "https://github.blog"],
 "maxItemsPerFeed": 25,
 "includeContent": false
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/rss-feed-scraper').call({
 feedUrls: ['https://arstechnica.com', 'https://blog.rust-lang.org/feed.xml'],
 sinceDate: '2026-08-01',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) console.log(`${item.publishedAt}, ${item.feedTitle}: ${item.title}`);
```

### Limits and behaviour

- **Feeds only carry a window.** Most publishers expose their 10-50 most recent items and nothing older. This Actor returns what the feed contains; it is not an archive crawler.
- **Formats are a mess, and that is handled.** RSS 0.91, RSS 1.0 (RDF), RSS 2.0, Atom 1.0 and JSON Feed 1.x all normalise to the same record. CDATA, double-encoded entities, `content:encoded`, `media:*` and `itunes:*` extensions and relative links are all dealt with.
- **Dates are normalised.** RFC-822 (`Thu, 06 Aug 2026 03:56:29 GMT`) and ISO-8601 both come out as ISO-8601 UTC. A date the parser cannot read becomes `null` rather than a wrong guess.
- **One failure never kills the run.** A dead host, a 404 or a page with no discoverable feed is logged, recorded in `RUN_SUMMARY.failures`, and the next source is read. The Actor only errors out if *every* source fails.
- **Transient errors are retried.** 429 and 5xx get three attempts with backoff, honouring `Retry-After` when the server sends one.
- **One request per second** between sources, with a 45-second timeout each.
- **Public data only.** No authentication, no personal data, no paywall circumvention.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Can it read a feed behind a login?** No. Authenticated and private feeds are out of scope; you will get an "access denied" failure in `RUN_SUMMARY`.

**What if a site has several feeds?** Discovery follows the first advertised one. If you want a specific category feed, pass its URL directly, you can pass as many as you like in one run.

**Why is `contentText` the same as `summary` for some feeds?** Because that feed only publishes a summary. Hacker News and most link blogs never send a body; nothing is being dropped.

**Why is `author` null?** Plenty of RSS feeds simply omit `dc:creator` and `<author>`. The Actor does not guess.

**Can I get only new items?** Run on a schedule and diff on `itemId`, or set `sinceDate` to your last run time. `includeContent: false` makes those sweeps much cheaper.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `feedUrls` (type: `array`):

Feed URLs to read. A site homepage works too: if the URL is not itself a feed, the page's <link rel="alternate" type="application/rss+xml"> tags are followed, and failing that the conventional paths (/feed, /rss.xml, /atom.xml, /index.xml) are tried. A URL that resolves to no readable feed is recorded in RUN\_SUMMARY.failures and the run continues.

## `maxItemsPerFeed` (type: `integer`):

Cap on saved items per feed, applied after filtering. Most feeds only publish their 10-50 most recent items anyway, so leave this at 0 unless you are billing-sensitive or reading very long archives.

## `sinceDate` (type: `string`):

Keep only items published after this moment. Accepts a plain date ('2026-01-01') or a full ISO-8601 timestamp ('2026-01-01T00:00:00Z'). Items whose feed gives no publish date are never dropped by this filter. Leave empty for everything the feed carries.

## `keywordFilter` (type: `array`):

Keep only items whose title, summary or categories contain one of these terms (case-insensitive, OR-combined). Empty keeps everything. Matching happens on the summary, not the full body, so a term buried deep in an article will not match.

## `includeContent` (type: `boolean`):

Include the full article body in 'contentText'. from <content:encoded> on RSS, <content> on Atom or 'content\_html' on JSON Feed. Turn off for a much smaller dataset when you only need headlines and links.

## `stripHtmlTags` (type: `boolean`):

On (default), bodies are delivered as clean plain text and 'contentHtml' is null. Off, the publisher's original markup is kept in 'contentHtml' as well. useful if you need the images and links inside an article.

## Actor input object example

```json
{
  "feedUrls": [
    "https://github.blog/feed/",
    "https://arstechnica.com"
  ],
  "maxItemsPerFeed": 0,
  "sinceDate": "2026-01-01",
  "keywordFilter": [
    "kubernetes",
    "postgres"
  ],
  "includeContent": true,
  "stripHtmlTags": true
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://news.ycombinator.com/rss",
        "https://blog.rust-lang.org/feed.xml",
        "https://daringfireball.net/feeds/json"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/rss-feed-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "feedUrls": [
        "https://news.ycombinator.com/rss",
        "https://blog.rust-lang.org/feed.xml",
        "https://daringfireball.net/feeds/json",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/rss-feed-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://news.ycombinator.com/rss",
    "https://blog.rust-lang.org/feed.xml",
    "https://daringfireball.net/feeds/json"
  ]
}' |
apify call arman-bd/rss-feed-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/rss-feed-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/eR3qMR7ABQdpHHm1M/builds/ciCSedcfaJHqD7co0/openapi.json