Feed Reindex Trigger — Multi-Feed RSS/Atom Monitor for RAG & Agents

Point it at as many RSS/Atom feeds as you track, and get back only the items that are actually new since the last check — deduped by GUID, returned as clean chunk-ready Markdown, not raw per-item JSON you have to post-process yourself.

The problem

Keeping a RAG index or an agent's context fresh from multiple content sources means polling a lot of feeds on a schedule. Most RSS actors solve half the problem: they'll parse one feed into JSON. They don't dedupe against what you already ingested, they don't batch many feeds behind one call, and they charge per item on every single poll -- including items you already have -- because they don't track state between runs at all. You end up building your own dedupe and diffing layer on top of someone else's parser, or paying for the same items again and again on every schedule tick.

Why not just use another RSS actor?

Because the Store's existing RSS actors fall into one of three buckets, and none of them solve the actual RAG/agent ingestion problem:

• Plain parsers turn a feed into JSON and charge per item extracted -- every run, whether or not you've seen that item before. Point one at a fast-moving feed on an hourly schedule and you're re-buying the same headlines all week.
• AI-enrichment bolt-ons add sentiment or summary scoring on top of RSS parsing, at 10-20x the per-item price of a plain parser -- useful if you want that specific analysis, wasteful if all you need is clean text to embed.
• Generic MCP wrappers expose feed-fetching as a raw tool call with no opinion about incremental state, dedupe, or output shape -- which is exactly why one of these launched with textbook-perfect MCP framing and still has zero users five weeks later. Being MCP-callable isn't the differentiator; being useful to call is.

This actor does the one thing none of them do: track what you've already seen, per feed, across runs, and only ever return what's actually new -- in the format a RAG pipeline wants to consume, not the format a parser happened to produce.

What it does

1. Accepts a list of feed URLs (RSS 2.0, Atom, RSS 1.0/RDF) in a single call -- no separate actor run per feed.
2. Fetches and parses each feed, and compares every item's GUID (falling back to a link+title hash when a feed omits GUIDs) against a named, persistent snapshot for that feed -- not the default key-value store, which is purged between runs and would silently make every run look like the first one.
3. Returns only items that are new or materially changed since the last snapshot, as clean Markdown with front-matter metadata (source feed, title, link, published date, status), ready to chunk and embed.
4. Optionally filters by keyword before charging, so a broad feed can be narrowed to what actually matters to your pipeline.
5. Optionally posts a webhook the moment new items are found, for push-based ingestion instead of poll-and-check.
6. Runs as a Standby HTTP/MCP endpoint: call it synchronously and get structured Markdown back inline, built for agents and schedulers to call directly. One unreachable or malformed feed in a multi-feed call is reported inline as an error entry for that feed only -- it never fails the whole call.

Who it's for

• RAG pipeline builders monitoring many content sources (blogs, changelogs, news feeds, docs feeds) who need only what's new, in embeddable form, not a full re-parse every time.
• AI agents (via Apify's MCP server / x402 agentic payments) that need a bounded, structured "what's new" tool rather than a raw feed dump.
• Newsletter and content-curation operators who want deduped, clean source material across many feeds in one call instead of stitching together several single-feed actors.
• Anyone who's paid twice for the same RSS item because their current tool has no memory between runs.

How recurring / scheduled checks work

Like Sitemap Reindex Trigger, this runs as an always-ready Standby endpoint rather than a scheduled batch job -- point your own scheduler (cron, GitHub Actions, Zapier/Make, or an agent's own task loop) at it on whatever cadence you want. Memory and idle-timeout are right-sized for a non-browser HTTP tool so the container doesn't sit warm (and billing you) between polls.

Use with AI agents via MCP / Standby

Call it synchronously with a list of feed URLs and get back only the new/updated items since your last call for each one, as Markdown blocks tagged with their source feed and metadata:

GET /?feedUrl=https://example.com/feed.xml
GET /?feedUrls=https://a.com/feed.xml,https://b.com/rss&snapshotGroup=myAgent&keywords=launch,release

Example real response, first call against hnrss.org's front-page feed (baseline run -- see "What happens on the very first check" in the FAQ below):

{
  "type": "summary",
  "feedUrl": "https://hnrss.org/frontpage",
  "feedTitle": "Hacker News: Front Page",
  "snapshotGroup": "default",
  "isFirstRun": true,
  "stats": { "totalItems": 20, "newItems": 0, "updatedItems": 0, "filteredOut": 0 },
  "newItems": [],
  "timestamp": "2026-07-07T06:38:32.221Z"
}

Example agent prompts once connected via Apify's MCP server:

• "Check our tracked blogs and changelogs for anything new since yesterday and summarize it."
• "Monitor these five feeds and only tell me about posts mentioning 'pricing' or 'launch'."
• "Set up a daily check on our competitor's changelog feed and post new entries to Slack."

Engineered for real usage economics

Like Sitemap Reindex Trigger, this actor runs at 256 MB with a 60-second idle timeout rather than Apify's 4 GB Standby default -- at default memory an always-on container would cost roughly $0.80/hour just for existing, regardless of traffic. Right-sized, a real Standby container lifecycle (spin up on request, handle it, idle out) measured at ~1 minute cost $0.001 in platform compute on a live test run. You pay for the brief window around actual use, not for a container sitting warm between polls.

Input

Field	Description
feedUrl / feedUrls	Single feed (agent convenience) or a batch array, capped by maxFeeds (default 25).
snapshotGroup	Track the same feed on two independent schedules without their state colliding. Defaults to "default".
keywords	Case-insensitive substring filter on title/content, applied before charging.
maxItemsPerFeed	Cap on items read per feed per run (default 100).
maxFeeds	Cap on feeds processed in one call (default 25).
webhookUrl / webhookOnEmptyChanges	Optional push notification when new/updated items are found.

Output reference

Field	Description
isFirstRun	true the first time a feed/snapshotGroup pair is checked. Baseline only, seeds the snapshot -- real deltas start on the next call.
stats.totalItems	Total items found in the feed this call.
stats.newItems / updatedItems	Counts of genuinely new items (never seen before) and updated items (same GUID, changed content).
stats.filteredOut	Items excluded by the keywords filter before charging.
newItems[].status	"new" or "updated" for each returned item.
newItems[].markdown	YAML front-matter (feed, title, link, publishedAt, author, categories, status) followed by the item body as clean, chunk-ready Markdown.
newItems[].dedupeKey	The stable key used to track this item across calls -- the feed's GUID, or a fallback hash when a feed omits one.
webhook	Present if webhookUrl was set: whether delivery was attempted and whether it succeeded.
type: "error" entries	In a feedUrls batch call, a feed that failed to fetch or parse appears as its own { "type": "error", "input": "<url>", "error": "..." } entry rather than failing the whole call.

Pricing

Charged per feed checked and per genuinely new/changed item returned -- never per item on a feed that hasn't moved since your last call. No reserved default events left over from the pricing wizard; every charge event matches something the code actually does (run-start, feed-checked, item-emitted).

FAQ

How is this different from a plain RSS-to-JSON actor? Plain parsers give you every item, every time, and leave dedupe and "what's new since last time" entirely to you. This tracks state between calls and only returns (and charges for) items that are actually new or changed.

How is this different from other feed monitors on the Store? The incumbents we found are either plain per-item parsers with no memory between runs, AI-enrichment bolt-ons at several times the price for analysis most RAG pipelines don't need, or a generic MCP wrapper with no opinion on incremental state or output shape. This combines multi-feed batching, GUID dedupe against a persistent snapshot, and RAG-ready Markdown output in one call.

Does this replace Apify's Schedule feature? No -- see "How recurring checks work" above. Point your own scheduler at the Standby URL instead of using Apify's Schedule feature.

What happens on the very first check of a new feed? It saves a baseline snapshot and reports zero new/updated items by design -- there's nothing to compare against yet. Real deltas appear from the second check onward.

What if one feed in a feedUrls batch call is unreachable or malformed? That feed's entry in the response is a { "type": "error", ... } object with the error message; every other feed in the same call is still processed and returned normally.

Is this legal? Yes. RSS and Atom feeds are public files intentionally published by site owners specifically so readers and tools can subscribe to them. This actor only reads public feed URLs you provide -- it doesn't bypass authentication or access private data.