# RSS to Markdown (full-text feed reader) (`vellumlabs/rss-to-markdown`) Actor

Fetch RSS, Atom and JSON feeds, then extract each entry's full article as clean Markdown (title, author, published, content, word count, content hash). Dedupes across runs with a named store. Ideal for LLM wikis, newsletters and monitoring. Pay per entry.

- **URL**: https://apify.com/vellumlabs/rss-to-markdown.md
- **Developed by:** [Vellum Kasane](https://apify.com/vellumlabs) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 article extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RSS to Markdown (full-text feed reader)

Turn any **RSS, Atom or JSON Feed into clean, LLM-ready Markdown**. Give the Actor a list of feed URLs and it returns one record per entry with the feed metadata (title, link, author, date, categories, summary) **plus the full article extracted from the entry's page** as GitHub-flavoured Markdown, with word count and content hash. Remember what was already exported with a state key, and scheduled runs return only new entries.

Most feed readers on Apify Store stop at the feed XML. This one fetches the articles:

- **Full text, not teasers.** Each entry's page is fetched (plain HTTP, no browser) and the main article is extracted with a readability-style pipeline: known CMS selectors (Ghost, WordPress, Substack, Medium-style, docs frameworks), then a text-density fallback, then chrome removal (navigation, share bars, related posts, newsletter boxes, comments, cookie banners). Feeds that embed the full post (`content:encoded`, Atom `content`, JSON `content_html`) are used directly when fetching is off or the page fails.
- **Every feed flavour.** RSS 2.0, RSS 1.0 / RDF, Atom 1.0 and JSON Feed 1.x, detected automatically. Namespaced fields (`dc:creator`, `content:encoded`, `dc:date`) are handled.
- **Dedupe across runs.** Set **State key** and the Actor keeps a hash of every exported URL in a named key-value store; the next run skips them. Tracking parameters (`utm_*`, `fbclid`, ...) and fragments are stripped before hashing, so the same article behind different links is stored once.
- **Date window.** `sinceDays` keeps only entries newer than N days.
- **Cheap and fast.** 20 entries from two feeds, with full-text extraction, in about a minute on 256 MB.

### Who is it for

- **LLM and RAG pipelines** that want article bodies, not `<description>` snippets.
- **Newsletter and digest builders** (n8n, Make, Zapier via the Apify integrations) that need "new since last run" semantics.
- **Knowledge bases**: push new posts from a set of blogs into Obsidian, Notion, a Claude Code wiki or a vector index on a schedule.
- **AI agents** using the [Apify MCP server](https://mcp.apify.com): call the Actor with a feed URL and read the Markdown from the dataset.

### Input

| Field                | Type     | Default  | Notes                                                                                             |
| -------------------- | -------- | -------- | ------------------------------------------------------------------------------------------------- |
| `feedUrls`           | string\[] | required | RSS / Atom / JSON Feed URLs.                                                                      |
| `maxItemsPerFeed`    | integer  | 50       | Newest entries kept per feed after filtering and dedupe. Caps cost.                               |
| `fetchFullContent`   | boolean  | `true`   | Fetch each entry's page and extract the article. Off = feed data only.                            |
| `sinceDays`          | integer  | 0        | `0` = no date filter. Undated entries are always kept.                                            |
| `stateKey`           | string   | `""`     | Name of a key-value store that remembers exported URLs between runs, e.g. `rss-state-newsletter`. |
| `contentSelector`    | string   | `""`     | CSS selector of the article body. Empty = auto-detect.                                            |
| `removeSelectors`    | string\[] | `[]`     | Extra selectors stripped from the article.                                                        |
| `respectRobotsTxt`   | boolean  | `true`   | Skip article URLs disallowed by robots.txt (feed data is still stored).                           |
| `maxConcurrency`     | integer  | 5        | Parallel article requests.                                                                        |
| `proxyConfiguration` | object   | no proxy | Enable Apify Proxy only if a site blocks you.                                                     |

Example input:

```json
{
    "feedUrls": ["https://blog.apify.com/rss/", "https://hnrss.org/frontpage"],
    "maxItemsPerFeed": 10,
    "fetchFullContent": true,
    "stateKey": "rss-state-demo"
}
```

### Output

One dataset item per entry:

```json
{
    "feed_url": "https://blog.apify.com/rss/",
    "feed_title": "Apify Blog",
    "feed_format": "rss",
    "id": "6aa24918b98a56000133a386",
    "title": "Apify vs. Exa: which gives AI agents better access to web data?",
    "url": "https://blog.apify.com/apify-vs-exa-comparison",
    "published": "2026-09-21T10:39:33.000Z",
    "updated": "",
    "author": "Egop Gogo-Job",
    "categories": ["AI agents", "Tool comparisons", "MCP"],
    "summary": "Choose your AI agent’s web tools based on actual evidence. Compare what Apify and Exa returned, what they missed, how long each task took, and how much it cost.",
    "content_markdown": "An [AI agent](https://en.wikipedia.org/wiki/AI_agent) can find the right webpage and still miss the data it needs.\n\nSome details are embedded in a page’s code...",
    "content_source": "page",
    "word_count": 2935,
    "content_hash": "e329100b4488efce1905f0d84a97522cb6806a8737553b1536b2ab35026c642c",
    "url_hash": "d076f53eee76cd7b6fce13af8f43862fa3e10bdfa608b55e2020fd074db27af3",
    "page_title": "How much web data can your AI agent unlock with Apify or Exa?",
    "lang": "en",
    "image": "https://storage.ghost.io/.../Apify_vs_EXA.webp",
    "extracted": true,
    "extraction_error": null,
    "fetched_at": "2026-09-22T14:41:37.157Z"
}
```

| Field              | Meaning                                                                                                                                |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| `content_source`   | `page` (article extracted from the entry URL), `feed` (converted from the feed's embedded HTML) or `null` (summary only).              |
| `extracted`        | `true` only when the body came from the page. This decides which billing event applies.                                                |
| `extraction_error` | Why the page was not used: fetch failed, robots.txt, non-HTML URL, or no article text found. The entry is still stored with feed data. |
| `content_hash`     | SHA-256 of `content_markdown` (or the summary), for change detection.                                                                  |
| `url_hash`         | SHA-256 of the normalised URL; the dedupe key stored under **State key**.                                                              |

`SUMMARY.json` in the key-value store lists per-feed statistics (entries seen, kept, skipped as already exported, skipped as too old, errors) and run totals. With a **State key**, the named store holds `SEEN_URLS` (`{ hashes: { <url_hash>: <fetched_at> }, updatedAt }`, capped at 100,000 newest entries).

### Pricing

This Actor uses **pay-per-event** pricing. Every stored entry is charged exactly once, at one of two rates:

| Event               | Price                                   | When                                                                                                             |
| ------------------- | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `article-extracted` | **$0.002 per entry** ($2 per 1,000)     | The article body was fetched and extracted from the entry's page (`extracted: true`).                            |
| `entry-exported`    | **$0.0005 per entry** ($0.50 per 1,000) | Feed data only: `fetchFullContent` off, feed-embedded content, non-HTML links, robots.txt skips or failed pages. |
| Actor start         | $0.005 per run                          | Apify's standard start event; covers the first seconds of compute.                                               |

A daily run over 10 blogs that yields 30 new full-text articles costs about **$0.07**. Entries skipped by the date filter or the dedupe state are never charged. Set **Max total charge per run** to hard-cap spend; the Actor stops gracefully before exceeding it. Platform usage (compute, storage) is included in the event prices.

### Tips

- **Scheduling**: create a Schedule with a fixed `stateKey`; each run stores only entries not seen before. Use a different key per project.
- **First run of a big archive**: combine `sinceDays: 30` with `maxItemsPerFeed` so the first run does not export years of history.
- **Wrong body picked up**: pass `contentSelector` (e.g. `.post-content`, `article .entry`) and add stray blocks to `removeSelectors`.
- **Paywalled or JavaScript-only sites**: the Actor fetches HTML over HTTP and does not run JavaScript; such entries fall back to the feed's content or summary and are charged at the lower `entry-exported` rate.
- **Podcasts / media feeds**: enclosure links are ignored; the entry's `link` is used. Non-HTML links (`.pdf`, `.mp3`, ...) are stored as feed data only.

### Limits

- Feeds are fetched with a 30 s timeout and two retries (back-off on 429/5xx). A feed that fails is reported in `SUMMARY.json` and does not fail the run.
- Article pages: 45 s navigation timeout, 2 retries, `respectRobotsTxt` on by default.
- `maxItemsPerFeed` up to 5,000; dedupe state holds the 100,000 most recent URLs per state key.

### Verified

Local run on 2026-09-22 with `apify run` (Apify CLI 1.10.0, Apify SDK 3.7.2, Crawlee 3.18.1, Node 24), prefill input (`https://blog.apify.com/rss/` + `https://hnrss.org/frontpage`, `maxItemsPerFeed: 10`, `fetchFullContent: true`, concurrency 5):

- 2 feeds parsed (RSS 2.0, 15 and 20 entries), **20 entries stored, 20 articles extracted** from 20 page requests (20 succeeded, 0 failed), runtime 71.8 s (one page timed out at 45 s and succeeded on retry; without it about 25 s). Word counts 274 to 5,116 per article; authors and publish dates came from the feed for all 20.
- Output spot-checked: Ghost posts (blog.apify.com) extracted from `.post-content` with headings, links and images, no related-posts / newsletter / share chrome; HN front-page links (news sites, personal blogs, a forum) extracted via known selectors or the text-density fallback.
- Pay-per-event path tested with `ACTOR_TEST_PAY_PER_EVENT=1 ACTOR_MAX_TOTAL_CHARGE_USD=5` (local test price $1/event) and `stateKey: rss-state-test`: 5 entries stored and charged as `article-extracted`, then "Run budget will be exhausted by the next entry - stopping gracefully". A second run with the same state key reported `skippedSeen: 5` and exported the next 5 entries; the named store ended with 10 hashes.
- Unit tests: `npm test` (19 tests with `node --test`: format detection, RSS 2.0 / RDF / Atom / JSON Feed parsing on real and synthetic fixtures, URL normalisation and hashing, article extraction on a real Ghost page, density fallback, explicit selector, feed-HTML conversion). `eslint` and `apify validate-schema` pass.

### Changelog

- **0.1.1** (2026-09-23): beehiiv newsletters now return the whole post body (previously only one content block was extracted).
- **0.1.0** (2026-09-22): initial release. RSS 2.0 / RDF / Atom / JSON Feed, full-text extraction, date window, cross-run dedupe via named key-value store, pay-per-event billing with two events.

### Support

Open an issue on the Actor's **Issues** tab. Issues are answered within one business day; bug fixes ship as new builds without changing the input schema.

Made by Vellum Labs.

# Actor input Schema

## `feedUrls` (type: `array`):

RSS / Atom / JSON Feed URLs. Each feed is fetched once per run.

## `maxItemsPerFeed` (type: `integer`):

Newest entries kept per feed after date filtering and deduplication. Each stored entry is one billing event, so this caps your cost.

## `fetchFullContent` (type: `boolean`):

Fetch each entry's page and extract the main article as Markdown (`article-extracted` event). When off, only feed data is stored (`entry-exported` event); feeds that embed the full post (`content:encoded`) are still converted to Markdown.

## `sinceDays` (type: `integer`):

`0` = no date filter. Entries without a date are always kept.

## `stateKey` (type: `string`):

Name of a key-value store used to remember exported entry URLs, e.g. `rss-state-newsletter`. Leave empty to only deduplicate within the run. With a state key, scheduled runs return only entries not seen before.

## `contentSelector` (type: `string`):

CSS selector of the article body on the entry pages. Leave empty to auto-detect (Ghost, WordPress, Substack, Medium-style layouts, `article`, `main`, plus a text-density fallback).

## `removeSelectors` (type: `array`):

Extra elements stripped from the article before conversion, on top of the built-in list (navigation, share bars, related posts, newsletter boxes, comments, cookie banners).

## `respectRobotsTxt` (type: `boolean`):

Skip article URLs disallowed by the site's robots.txt (the feed data is still stored).

## `maxConcurrency` (type: `integer`):

Parallel article requests.

## `proxyConfiguration` (type: `object`):

Proxy settings for article fetching. Most sites need no proxy; enable Apify Proxy only if you get blocked.

## Actor input object example

```json
{
  "feedUrls": [
    "https://blog.apify.com/rss/",
    "https://hnrss.org/frontpage"
  ],
  "maxItemsPerFeed": 10,
  "fetchFullContent": true,
  "sinceDays": 0,
  "stateKey": "",
  "contentSelector": "",
  "removeSelectors": [],
  "respectRobotsTxt": true,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `entries` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://blog.apify.com/rss/",
        "https://hnrss.org/frontpage"
    ],
    "maxItemsPerFeed": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("vellumlabs/rss-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "feedUrls": [
        "https://blog.apify.com/rss/",
        "https://hnrss.org/frontpage",
    ],
    "maxItemsPerFeed": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("vellumlabs/rss-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://blog.apify.com/rss/",
    "https://hnrss.org/frontpage"
  ],
  "maxItemsPerFeed": 10
}' |
apify call vellumlabs/rss-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,vellumlabs/rss-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/a65IrzFZp4NAamZ2T/builds/aPxS9wjHfpfmycaIG/openapi.json