# Convert Any Website to RSS Feed (`thescrapelab/any-website-to-rss-feed`) Actor Turn blogs, news pages, job boards, product listings, directories, and sitemaps into RSS feeds, JSON Feed, and structured datasets with change detection. - **URL**: https://apify.com/thescrapelab/any-website-to-rss-feed.md - **Developed by:** [Inus Grobler](https://apify.com/thescrapelab) (community) - **Categories:** SEO tools, News, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Any Website to RSS Feed **Website to RSS Feed Generator for blogs, news pages, job boards, product listings, directories, and sitemaps.** Turn almost any public website into an RSS feed, JSON Feed, and structured Apify dataset. Any Website to RSS Feed is a lightweight website to RSS feed generator for blogs, news pages, job boards, product listings, directories, category pages, and sitemap-backed websites. Use it when a site does not offer a clean RSS feed, when you want a normalized feed from many different websites, or when you need to detect new and changed items over time. The Actor is designed to be cost-aware by default. It looks for existing RSS, Atom, and JSON feeds first, then tries static HTML extraction, and only uses browser rendering or optional AI selector discovery when needed. ### SEO Title Convert Any Website to RSS Feed | Blogs, News, Jobs ### SEO Description Create RSS feeds from blogs, news pages, job boards, product listings, directories, and sitemaps. Export JSON Feed and structured datasets with lightweight change detection on Apify. ### Why People Use It - Turn websites without RSS into feeds for Slack, email, Zapier, Make, or custom automation - Monitor job boards, product catalogs, newsroom sections, and directory pages - Publish normalized RSS and JSON Feed outputs from inconsistent websites - Detect new and changed content across scheduled runs without building a custom scraper ### What You Can Do With It - Create an RSS feed from almost any public website - Convert blogs, listings, directories, and job boards into structured feed items - Generate both RSS 2.0 and JSON Feed from the same run - Export feed items to an Apify dataset for automation and analysis - Detect new, changed, and unchanged items across repeated runs - Use custom CSS selectors for difficult websites - Use optional OpenRouter selector discovery when normal extraction is weak ### Common Use Cases - Website to RSS feed conversion - Blog to RSS feed generation - Job board monitoring - Product listing monitoring - News and article aggregation - Directory and category page tracking - Sitemap-based content discovery - Lightweight website change monitoring for feed publishing - RSS feed generation for automation tools, newsletters, and dashboards ### How It Works For each URL you provide, the Actor tries these approaches: 1. Finds existing RSS, Atom, or JSON feeds 2. Checks common feed URLs such as `/feed`, `/rss.xml`, and `/atom.xml` 3. Discovers sitemap URLs from `robots.txt` and `sitemap.xml` 4. Extracts repeated cards or listing items from static HTML 5. Uses your custom CSS selectors when provided 6. Falls back to browser rendering when JavaScript is required 7. Optionally uses OpenRouter to infer selectors for repeated listings 8. Stores RSS, JSON Feed, dataset rows, and a run summary ### Quick Start Paste one or more public URLs into `startUrls`. ```json { "startUrls": [ { "url": "https://example.com/blog" } ] } ```` ### Best Starting Setup For most websites, this is the best place to start: ```json { "startUrls": [ { "url": "https://example.com/blog" } ], "mode": "auto", "maxPages": 25, "maxItems": 100 } ``` Use `pageList` for category pages, search results, job boards, and product grids. Use `customSelectors` only when a site has an unusual layout and automatic extraction is weak. For a listing page, such as jobs, products, articles, or directory cards: ```json { "startUrls": [ { "url": "https://example.com/jobs" } ], "mode": "pageList", "maxPages": 25, "maxItems": 100 } ``` For a page where you already know the selectors: ```json { "startUrls": [ { "url": "https://example.com/products" } ], "mode": "customSelectors", "customSelectors": { "itemSelector": ".product-card", "titleSelector": ".product-title", "urlSelector": "a", "summarySelector": ".product-summary", "imageSelector": "img" } } ``` ### Input Options - `startUrls`: public website URLs to convert into feed items - `mode`: choose automatic discovery, RSS-only discovery, sitemap mode, page-list extraction, or custom selectors - `maxPages`: maximum pages to fetch - `maxItems`: maximum items to return - `renderJavaScript`: use browser rendering for JavaScript-heavy pages - `includeUrlPatterns`: keep only URLs matching these patterns - `excludeUrlPatterns`: remove URLs matching these patterns - `customSelectors`: CSS selectors for manual extraction - `changedItemPolicy`: include changed items, exclude them, or output only changed items Technical settings such as crawl depth, browser fallback, deduplication, and debug reporting stay on safe built-in defaults so the input stays simple. ### Output The Actor writes feed items to the dataset and stores feed files in the run key-value store. Dataset items include: ```json { "title": "Frontend Engineer", "url": "https://example.com/jobs/frontend-engineer", "canonicalUrl": "https://example.com/jobs/frontend-engineer", "summary": "Frontend role for a fast-growing product team.", "publishedAt": "2026-05-13T09:00:00.000Z", "updatedAt": null, "imageUrl": "https://example.com/images/frontend.png", "sourcePageUrl": "https://example.com/jobs", "sourceSite": "https://example.com", "contentHash": "sha256...", "isNew": true, "isChanged": false, "previousHash": null, "discoveryMethod": "html_repeated", "extractionMethod": "static_html", "confidence": 0.86, "scrapedAt": "2026-05-13T12:00:00.000Z" } ``` Key-value store records: - `RSS_XML`: generated RSS 2.0 feed - `JSON_FEED`: generated JSON Feed - `RUN_SUMMARY`: totals, failed URLs, and extraction flags - `DEBUG_REPORT`: optional debug details when troubleshooting is enabled internally ### RSS and JSON Feed The generated RSS feed includes channel metadata and item fields such as title, link, guid, publication date, and description when available. The generated JSON Feed includes feed metadata and item fields such as id, URL, title, summary, publication date, and image when available. Descriptions and summaries are optional because many listing pages do not expose full excerpts. When possible, the Actor tries to enrich missing summaries from page metadata and detail pages while staying within your page limits. ### New and Changed Items The Actor can track items across runs using a named state store. This lets you tell whether an item is: - new - changed - unchanged This is useful for scheduled RSS generation, content alerts, product monitoring, job board monitoring, and feed publishing workflows. ### Example Saved Tasks These are good starting points for production tasks in Apify. #### Monitor a News Section ```json { "startUrls": [ { "url": "https://example.com/news" } ], "mode": "auto", "maxPages": 20, "maxItems": 50, "changedItemPolicy": "include" } ``` #### Track a Job Board ```json { "startUrls": [ { "url": "https://example.com/jobs" } ], "mode": "pageList", "maxPages": 30, "maxItems": 100, "changedItemPolicy": "onlyChanged" } ``` #### Follow a Product Listing Page ```json { "startUrls": [ { "url": "https://example.com/products" } ], "mode": "pageList", "maxPages": 20, "maxItems": 100, "includeUrlPatterns": [ "/products/" ] } ``` ### Optional AI Selector Discovery Some websites have unusual markup. When needed, the Actor can use OpenRouter to infer CSS selectors for repeated cards or listing items. The AI is only used for selector discovery. It is not used to summarize content, rewrite text, or analyze the full website. To use this feature, set `OPENROUTER_API_KEY` as a secret environment variable in Apify. Do not put the key in your input JSON. ### Pricing and Resource Tips - Start with `mode: "auto"` - Keep `renderJavaScript` off unless the site requires it - Use `maxPages` and `maxItems` to control cost - Use `includeUrlPatterns` for large websites and sitemaps - Use `customSelectors` for pages with predictable layouts - Use OpenRouter only as a fallback unless a website clearly needs selector inference Practical guidance: - Small blog or feed-backed website: usually `maxPages: 10-25` - News section or job board: usually `maxPages: 20-40` - Large sitemap or directory: start with `maxPages: 25` and narrow with `includeUrlPatterns` - Turn on `renderJavaScript` only for sites that clearly depend on client-side rendering If you are launching this as a scheduled production task, start conservative, review the first few runs, and only increase page limits when the extra items are worth the extra cost. ### Limitations - Works on public pages only - Does not log in to websites - Does not perform screenshot comparison - Does not generate AI summaries - Does not crawl deeply by default - Some complex websites may require custom selectors ### Troubleshooting - Empty dataset: try `mode: "customSelectors"` with manual selectors - Missing summaries: the source page may not expose descriptions or excerpts - JavaScript-heavy site: enable `renderJavaScript` - Large sitemap: reduce `maxPages` or use `includeUrlPatterns` - LLM fallback not running: check that `OPENROUTER_API_KEY` is set as a secret environment variable ### Launch Checklist Before going live with a scheduled task: 1. Run the Actor once with `mode: "auto"` and confirm the dataset items look correct. 2. Check `RSS_XML`, `JSON_FEED`, and `RUN_SUMMARY` in the key-value store. 3. Make sure `maxPages` and `maxItems` are low enough for the budget you want. 4. Add `includeUrlPatterns` if the site is broad and you only want one section. 5. If the page is JavaScript-heavy, enable `renderJavaScript`. 6. If extraction is weak, switch to `customSelectors` before reaching for AI fallback. 7. Schedule the task and review the second run to confirm `isNew` and `isChanged` behave as expected. ### Python API Example Use the Apify API from Python with `apify_client`: ```python import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_API_TOKEN"]) run_input = { "startUrls": [{"url": "https://example.com/blog"}], "mode": "auto", "maxPages": 25, "maxItems": 100, } run = client.actor("TheScrapeLab/any-website-to-rss-feed").call(run_input=run_input) dataset = client.dataset(run["defaultDatasetId"]) store = client.key_value_store(run["defaultKeyValueStoreId"]) items = list(dataset.iterate_items()) rss_xml = store.get_record("RSS_XML")["value"] json_feed = store.get_record("JSON_FEED")["value"] run_summary = store.get_record("RUN_SUMMARY")["value"] print(f"Run ID: {run['id']}") print(f"Items found: {len(items)}") print(f"New items: {run_summary['newItems']}") print(f"Changed items: {run_summary['changedItems']}") print(items[0]["title"] if items else "No items found") print(rss_xml[:200]) print(json_feed["title"]) ``` ### Who This Actor Is For This website to RSS feed generator is useful for marketers, growth teams, recruiters, publishers, analysts, developers, automation builders, and anyone who needs reliable feed data from public websites. # Actor input Schema ## `startUrls` (type: `array`): One or more public URLs to inspect. ## `mode` (type: `string`): How the Actor should approach discovery and extraction. ## `maxPages` (type: `integer`): Upper limit for fetched pages across the run. ## `maxItems` (type: `integer`): Upper limit for emitted feed items. ## `renderJavaScript` (type: `boolean`): Use Playwright for the first fetch instead of static HTML. ## `includeUrlPatterns` (type: `array`): Only keep items whose URLs match one of these patterns. ## `excludeUrlPatterns` (type: `array`): Drop items whose URLs match one of these patterns. ## `customSelectors` (type: `object`): Optional selectors for item extraction. ## `changedItemPolicy` (type: `string`): Control whether changed items appear in outputs. ## Actor input object example ```json { "startUrls": [ { "url": "https://example.com/blog" } ], "mode": "auto", "maxPages": 50, "maxItems": 100, "renderJavaScript": false, "changedItemPolicy": "include" } ``` # Actor output Schema ## `datasetItems` (type: `string`): No description ## `rssXml` (type: `string`): No description ## `jsonFeed` (type: `string`): No description ## `runSummary` (type: `string`): No description ## `debugReport` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://example.com/blog" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("thescrapelab/any-website-to-rss-feed").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://example.com/blog" }] } # Run the Actor and wait for it to finish run = client.actor("thescrapelab/any-website-to-rss-feed").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://example.com/blog" } ] }' | apify call thescrapelab/any-website-to-rss-feed --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=thescrapelab/any-website-to-rss-feed", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Convert Any Website to RSS Feed", "description": "Turn blogs, news pages, job boards, product listings, directories, and sitemaps into RSS feeds, JSON Feed, and structured datasets with change detection.", "version": "0.1", "x-build-id": "Jj2y3zBIwixvUfT7N" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/thescrapelab~any-website-to-rss-feed/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-thescrapelab-any-website-to-rss-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/thescrapelab~any-website-to-rss-feed/runs": { "post": { "operationId": "runs-sync-thescrapelab-any-website-to-rss-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/thescrapelab~any-website-to-rss-feed/run-sync": { "post": { "operationId": "run-sync-thescrapelab-any-website-to-rss-feed", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Start URLs", "type": "array", "description": "One or more public URLs to inspect.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "mode": { "title": "Mode", "enum": [ "auto", "rssDiscoveryOnly", "sitemap", "pageList", "customSelectors" ], "type": "string", "description": "How the Actor should approach discovery and extraction.", "default": "auto" }, "maxPages": { "title": "Max pages", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Upper limit for fetched pages across the run.", "default": 50 }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Upper limit for emitted feed items.", "default": 100 }, "renderJavaScript": { "title": "Render JavaScript", "type": "boolean", "description": "Use Playwright for the first fetch instead of static HTML.", "default": false }, "includeUrlPatterns": { "title": "Include URL patterns", "type": "array", "description": "Only keep items whose URLs match one of these patterns.", "items": { "type": "string" } }, "excludeUrlPatterns": { "title": "Exclude URL patterns", "type": "array", "description": "Drop items whose URLs match one of these patterns.", "items": { "type": "string" } }, "customSelectors": { "title": "Custom selectors", "type": "object", "description": "Optional selectors for item extraction.", "properties": { "itemSelector": { "type": "string", "title": "Item selector", "description": "CSS selector for each repeated item card.", "editor": "textfield" }, "titleSelector": { "type": "string", "title": "Title selector", "description": "CSS selector for the item title within each card.", "editor": "textfield" }, "urlSelector": { "type": "string", "title": "URL selector", "description": "CSS selector for the link element within each card.", "editor": "textfield" }, "dateSelector": { "type": "string", "title": "Date selector", "description": "CSS selector for the publication date element within each card.", "editor": "textfield" }, "summarySelector": { "type": "string", "title": "Summary selector", "description": "CSS selector for the summary or excerpt element within each card.", "editor": "textfield" }, "imageSelector": { "type": "string", "title": "Image selector", "description": "CSS selector for the image element within each card.", "editor": "textfield" } } }, "changedItemPolicy": { "title": "Changed item policy", "enum": [ "include", "exclude", "onlyChanged" ], "type": "string", "description": "Control whether changed items appear in outputs.", "default": "include" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```