Website to RSS Feed Generator

Convert any website into an RSS or JSON feed. Auto-detects content, supports JavaScript sites, includes keyword filtering and full-text extraction.

Table of Contents

• Features
• Quick Start
• Input Parameters
• Output
• Examples
• Use Cases
• Pricing
• Limitations
• Changelog

Features

• Smart Auto-Detection - Finds articles, posts, and news items automatically
• Playwright Support - Handles JavaScript-rendered sites (React, Vue, Next.js)
• Multiple Formats - Output as RSS 2.0, JSON Feed 1.1, or both
• Keyword Filtering - Whitelist or blacklist items by keyword
• Full Content Extraction - Fetch complete article text, not just excerpts
• Pay-Per-Use - No monthly subscription, pay only for what you use

Quick Start

Basic Usage

Just provide a URL. The actor auto-detects content items:

{
    "url": "https://techcrunch.com"
}

📡 Use as a Live RSS Feed (Most Common Workflow)

The actor saves feed.xml to the run's Key-Value Store. That file has a permanent public URL that any RSS reader can pull from.

3-step setup for a self-updating feed:

1. Run the actor once (with your URL + settings). Note the RSS Feed URL printed in the logs:

   https://api.apify.com/v2/key-value-stores/{kvsId}/records/feed.xml
   

2. Paste that URL into your RSS reader (Feedly, Inoreader, NewsBlur, Reeder, etc.). The reader will pull fresh content every time it checks for updates.

3. Set up a schedule in Apify Console → Schedules → Create new → Pick this actor → Choose frequency (daily, hourly, 6-hourly). Every run overwrites feed.xml so your reader sees the latest items.

Why this beats a one-shot run: you never have to re-run manually. Schedule once, forget about it — your RSS reader pulls on its own cadence from the always-fresh Apify URL.

JavaScript Sites

For React, Vue, or other JS-heavy sites:

{
    "url": "https://react-blog.com",
    "usePlaywright": true,
    "waitForSelector": "article"
}

With Keyword Filters

Include only items containing specific keywords:

{
    "url": "https://news.ycombinator.com",
    "keywordWhitelist": ["AI", "startup"],
    "keywordBlacklist": ["crypto"]
}

Input Parameters

Parameter	Type	Default	Description
url	string	required	URL of the page to convert
outputFormat	string	rss	Output format: rss, json, or both
usePlaywright	boolean	false	Enable for JavaScript-rendered sites
extractFullContent	boolean	false	Fetch full article text from each link
maxItems	integer	20	Maximum items to include (1-100)
feedTitle	string	auto	Custom feed title
feedDescription	string	auto	Custom feed description
keywordWhitelist	array	[]	Only include items with these keywords
keywordBlacklist	array	[]	Exclude items with these keywords
itemSelector	string	auto	CSS selector for item containers
titleSelector	string	auto	CSS selector for item titles
linkSelector	string	auto	CSS selector for item links
descriptionSelector	string	auto	CSS selector for descriptions
dateSelector	string	auto	CSS selector for dates
imageSelector	string	auto	CSS selector for images
waitForSelector	string	-	Element to wait for (Playwright only)
proxyConfiguration	object	-	Apify proxy settings

Output

The actor saves feeds to the run's key-value store:

File	Format	Written when	Description
feed.xml	RSS 2.0	outputFormat is rss or both	Standard RSS feed, works with all readers
feed.json	JSON Feed 1.1	outputFormat is json or both	Modern format, ideal for APIs
OUTPUT	JSON	always	Run metadata, extracted items, and the public feedUrls

The default outputFormat is rss, so a default run writes feed.xml (and OUTPUT) only. Each key-value store record has a permanent public URL — see Use as a Live RSS Feed above.

Dataset Output

Each feed item is also saved to the dataset with these fields:

Field	Type	Description
title	string	Item title
link	string	URL to the full article
description	string	Excerpt or summary
date	string	Publication date (ISO format)
image	string	Featured image URL
author	string	Author name

Examples

Monitor a Blog

{
    "url": "https://spiderontheweb.com",
    "maxItems": 10,
    "outputFormat": "rss"
}

News Aggregation with Filters

{
    "url": "https://news-site.com",
    "keywordWhitelist": ["technology", "science"],
    "keywordBlacklist": ["sports", "entertainment"],
    "maxItems": 50
}

Full Content for Reading

{
    "url": "https://medium.com/topic/programming",
    "usePlaywright": true,
    "extractFullContent": true,
    "outputFormat": "both"
}

Custom Selectors

When auto-detection doesn't work:

{
    "url": "https://custom-site.com",
    "itemSelector": ".post-item",
    "titleSelector": "h2.title",
    "linkSelector": "a.read-more",
    "descriptionSelector": ".excerpt",
    "dateSelector": ".date"
}

Use Cases

• Competitor Monitoring - Track competitor blogs and announcements
• News Aggregation - Combine multiple sources in your RSS reader
• Research Tracking - Follow publications, reports, and changelogs
• Price Monitoring - Watch product pages for updates
• Job Tracking - Monitor career pages for new postings

Pricing

This actor runs on Apify's pay-per-use model. You are billed for the platform compute (compute units) each run consumes, plus any optional Apify Proxy usage — there is no monthly subscription, so you pay only for the runs you trigger.

Cost per run depends on the target site, maxItems, and which options you enable: turning on Playwright (usePlaywright) or full-content extraction (extractFullContent) does more work and therefore costs more per run. Scheduling more frequent runs increases total spend proportionally.

For current rates and a live estimate, see the Pricing and Runs tabs in the Apify Console, and start with a small maxItems value to gauge your typical cost.

Limitations

• Single page only. The actor processes just the one url you provide — it does not follow pagination or crawl additional pages. All feed items come from that page.
• Heuristic auto-detection. With no custom selectors, items are found by a scoring heuristic. Some layouts won't be detected and may return 0 items; for those, supply custom CSS selectors (itemSelector, titleSelector, etc.).
• Empty results don't error. If nothing is detected, the run still finishes successfully: OUTPUT.success is false, no dataset items are pushed, and no feed.xml / feed.json is written. Check the run log for guidance.
• Conditional feed files. feed.xml is written only when outputFormat is rss or both; feed.json only when it is json or both. The default (rss) writes feed.xml only.
• JavaScript rendering is opt-in. Static HTML is read with Cheerio by default. JS-rendered sites need usePlaywright: true (there is also an automatic Playwright fallback if Cheerio finds 0 items and Playwright is available in the Docker image).
• Full-text extraction scope. When extractFullContent is enabled, the full article text is embedded in the RSS/JSON feeds and the OUTPUT record, but not in the dataset items.
• Best-effort fields. date falls back to the run time when it can't be parsed; image and author are empty strings when not present on the page.
• Access limits. Sites behind logins, paywalls, or anti-bot protection may block the request; configure proxyConfiguration to improve success rates.
• Item cap. Feeds and datasets are limited to maxItems (1–100, default 20).

Changelog

v2.5.0

• Logs a permanent public feed URL (key-value store) to paste into any RSS reader
• Persists feedUrls (rss/json) in the OUTPUT record for programmatic access

v2.3.0

• Automatic Playwright fallback when Cheerio finds 0 items (safely skipped if Playwright isn't available in the Docker image)
• Graceful handling when no url is provided (health-check safe) and a warning for placeholder domains

v2.1.0

• Added output schema for dataset
• Improved documentation

v2.0.0

• Added Playwright support for JavaScript sites
• Added JSON Feed output format
• Added keyword whitelist/blacklist filtering
• Added full content extraction
• Improved auto-detection scoring algorithm
• Added author extraction

v1.0.0

• Initial release with auto-detection and RSS output