# AI-Ready RSS Content Cleaner (Article to Markdown) (`piotrhl/ai-ready-rss-content-cleaner`) Actor

Ultra-lightweight RSS & Atom feed parser that scrapes full article text, removes boilerplate (ads, nav, footers), and converts clean content into AI/RAG-ready Markdown under 128 MB RAM.

- **URL**: https://apify.com/piotrhl/ai-ready-rss-content-cleaner.md
- **Developed by:** [Piotr Hładik](https://apify.com/piotrhl) (community)
- **Categories:** AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI-Ready RSS Content Cleaner (Full-Text Article to Markdown)

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-orange.svg)](https://apify.com)
[![Monetization](https://img.shields.io/badge/Monetization-Pay--Per--Event-brightgreen.svg)](https://apify.com)
[![RAM Footprint](https://img.shields.io/badge/Memory-%3C128MB-blue.svg)](https://apify.com)
[![Python 3.11+](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://python.org)
[![Engine](https://img.shields.io/badge/Engine-HTTP%2F2%20Fast%20Core-teal.svg)](https://httpx.automattic.com)

**AI-Ready RSS Content Cleaner** is an enterprise-grade, ultra-lightweight content ingestion actor designed for AI engineers, RAG system architects, newsletter curators, and market intelligence teams.

Most RSS feeds only provide short, truncated summaries or messy HTML snippets stuffed with ad banners, tracking pixels, and navigation clutter. This Actor reads any standard RSS or Atom feed, fetches the full target article, strips all boilerplate DOM elements using fast C-based parsing (`selectolax.lexbor`), and formats the article into clean, structured Markdown ready for immediate LLM processing.

***

### ⚡ Why Use AI-Ready RSS Content Cleaner?

- 🚀 **Full-Text Ingestion (Not Just Snippets)**: Bypasses feed truncation by following article links and extracting the complete article text.
- 🧹 **Zero Boilerplate & Ad Removal**: Automatically strips `<script>`, `<style>`, `<nav>`, `<footer>`, `<header>`, `<aside>`, cookie notices, social share widgets, and ads.
- 🤖 **Structured Markdown for LLMs & RAG**: Formats headings (`#`, `##`, `###`), blockquotes (`>`), bullet lists (`-`), and code blocks (` ``` `) cleanly so your LLM embeddings preserve semantic context.
- 💰 **Runs on Under 128 MB RAM**: Built with asynchronous HTTP/2 (`httpx`) and fast C-based parsing (`selectolax.lexbor`), eliminating heavy headless browsers (Puppeteer/Playwright) and slashing execution cost.
- 🏷️ **Pay-Per-Event (PPE) Pricing**: Pay only for the actual articles cleaned and saved ($0.0015 per article). No wasted credits on empty or duplicate feeds.

***

### 🔌 High-Impact Use Cases & Integrations

#### 1. Vector Search & RAG Ingestion (LangChain, LlamaIndex, Pinecone)

Feed real-time news and industry blogs into your knowledge base:

1. Schedule this Actor to poll feeds hourly via Apify Schedules or webhooks.
2. Pipe output records straight into Pinecone, Qdrant, Weaviate, or Chroma.
3. Pass `markdown_content` directly as the `page_content` chunk, with `author`, `published_date`, `word_count`, and `source_feed_url` as metadata.

#### 2. Autonomous Newsletter Automation (n8n, Make.com)

Automate your weekly curation newsletter without manual copy-pasting:

1. Connect this Actor to your target publication feeds (Substack, TechCrunch, arXiv, Hacker News).
2. Filter articles in n8n or Make.com by `word_count` or date.
3. Use OpenAI or Claude to summarize the clean Markdown and draft issue briefs straight into Mailchimp, Beehiiv, or Ghost.

#### 3. Executive Briefing & Market Intelligence

Track competitors and market shifts across 50+ industry blogs daily without getting bogged down by ad trackers or bloated webpage scripts.

***

### ⚙️ Input Configuration

The Actor provides a clean Standard Mode for quick extraction and an Advanced / Nerd Mode for fine-tuning network parameters.

#### Standard Mode

| Field | Type | Default | Description |
|---|---|---|---|
| `rss_urls` | Array | `["https://techcrunch.com/feed/"]` | List of RSS or Atom feed URLs to process. |
| `max_articles_per_feed` | Integer | `20` | Maximum number of recent articles to clean per feed (1 to 500). |

#### Advanced / Nerd Mode (Engine & Network Tuning)

| Field | Type | Default | Description |
|---|---|---|---|
| `concurrency` | Integer | `5` | Number of simultaneous HTTP workers (1 to 25). |
| `request_timeout_sec` | Integer | `20` | Timeout in seconds for HTTP requests. |
| `custom_user_agent` | String | `None` | Optional custom HTTP User-Agent string. |
| `proxy_configuration` | Object | `{"useApifyProxy": true}` | Apify Proxy settings to bypass anti-scraping firewalls. |

***

### 📊 Normalized Output Schema

Each processed item is emitted into the default Apify Dataset:

````json
{
  "source_feed_url": "https://techcrunch.com/feed/",
  "article_url": "https://techcrunch.com/2026/10/17/the-rise-of-local-ai-models",
  "title": "The Rise of Local AI Models",
  "author": "Dr. Jane Doe",
  "published_date": "Sat, 17 Oct 2026 12:00:00 GMT",
  "word_count": 482,
  "reading_time_minutes": 2.4,
  "markdown_content": "# The Rise of Local AI Models\n\nOver the last year, small language models running locally on edge hardware have grown exponentially.\n\n## Key Benefits\n\n- Zero latency network bottlenecks\n- Complete data privacy and sovereignty\n- Drastically reduced operational cost\n\n> Privacy is not an afterthought; it is a fundamental architecture requirement.\n\n```\ncurl -X POST http://localhost:11434/api/generate\n```\n\nIn conclusion, local AI transforms consumer and enterprise software forever.",
  "scraped_at": "2026-10-17T20:25:30.123456+00:00"
}
````

***

### 💰 Pay-Per-Event (PPE) Pricing Transparency

This Actor operates under Apify's **Pay-Per-Event (PPE)** model:

| Event | Price (USD) | Description |
|---|---|---|
| `apify-actor-start` | **$0.00005** | Charged once at container startup. |
| `article-cleaned` | **$0.0015** | Charged once per full article cleaned and converted to Markdown. |

#### Why PPE Saves Budget:

- Cleaning 100 full-text articles costs only **$0.15**.
- If a feed has no new posts or fails to resolve, you are not billed for idle browser compute.
- Automatic budget protection terminates the run gracefully when your user spending limit is hit.

***

### ❓ Frequently Asked Questions (FAQ)

#### What happens if an article is behind a hard paywall?

If the full HTML page cannot be reached or returns an access restriction, the Actor automatically falls back to the RSS item's `content` or `summary` payload, cleaning and formatting whatever text is available.

#### Can it handle Atom feeds as well as RSS?

Yes. The engine parses RSS 0.9x, RSS 2.0, Atom 1.0, and RDF feeds seamlessly.

#### How are word count and reading time calculated?

Word count counts whitespace-delimited tokens from the cleaned Markdown content. Reading time is calculated using the standard cognitive reading benchmark of 200 words per minute.

***

### 🛠️ Local Development & Testing

```bash
## Clone the repository
git clone https://github.com/your-org/ai-ready-rss-content-cleaner.git
cd ai-ready-rss-content-cleaner

## Install dependencies into your local virtual environment
.venv\Scripts\python.exe -m pip install -r requirements.txt

## Run the test suite with 100% mocked offline requests
.venv\Scripts\python.exe -m pytest -v

## Run locally using Apify CLI
apify run
```

***

### 📜 License & Compliance

Distributed under the Apache 2.0 License. Ensure scraping and feed consumption comply with feed publisher policies.

# Actor input Schema

## `rss_urls` (type: `array`):

List of RSS or Atom feed URLs to monitor and extract.

## `max_articles_per_feed` (type: `integer`):

Maximum number of recent articles to fetch and clean per feed.

## `concurrency` (type: `integer`):

Number of concurrent network workers for downloading feeds and articles.

## `request_timeout_sec` (type: `integer`):

HTTP request timeout in seconds.

## `custom_user_agent` (type: `string`):

Custom HTTP User-Agent header string.

## `proxy_configuration` (type: `object`):

Apify Proxy settings to bypass anti-scraping and rate limits.

## Actor input object example

```json
{
  "rss_urls": [
    {
      "url": "https://techcrunch.com/feed/"
    }
  ],
  "max_articles_per_feed": 20,
  "concurrency": 5,
  "request_timeout_sec": 20,
  "proxy_configuration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "rss_urls": [
        {
            "url": "https://techcrunch.com/feed/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("piotrhl/ai-ready-rss-content-cleaner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "rss_urls": [{ "url": "https://techcrunch.com/feed/" }] }

# Run the Actor and wait for it to finish
run = client.actor("piotrhl/ai-ready-rss-content-cleaner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "rss_urls": [
    {
      "url": "https://techcrunch.com/feed/"
    }
  ]
}' |
apify call piotrhl/ai-ready-rss-content-cleaner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,piotrhl/ai-ready-rss-content-cleaner"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0gNsR1KP1vNVmb7L4/builds/JfEeIlbyujzZTn7Ke/openapi.json