News & Blog Change Monitor

Monitor news sites and blogs for new or updated articles. The Actor prefers RSS/Atom feeds and falls back to blog index page crawling when needed. Each run compares discovered articles against state stored in the Apify key-value store from previous runs.

Features

• RSS-first monitoring — parses RSS 2.0 and Atom feeds, with automatic feed discovery from HTML <link rel="alternate"> tags
• Blog index fallback — extracts article links, titles, dates, and authors from common blog listing layouts
• Change detection — returns new, updated, or unchanged per article
• Persistent state — stores article fingerprints in the default KV store between scheduled runs
• Retries & rate limiting — configurable HTTP retries and per-domain request delay
• Run report — summary JSON saved to the OUTPUT key-value store record

Input

Field	Type	Required	Default	Description
sources	array	Yes	—	URLs to monitor. Each item can be a string URL or { url, mode } where mode is auto, rss, or blog
maxArticlesPerSource	integer	No	50	Max latest articles checked per source
includeUnchanged	boolean	No	false	Include unchanged articles in dataset output
maxRequestRetries	integer	No	3	HTTP retry count
requestDelayMs	integer	No	500	Delay between same-domain requests (ms)
maxRequestsPerCrawl	integer	No	1000	Safety cap on total HTTP requests

Example input

{
  "sources": [
    { "url": "https://blog.apify.com/", "mode": "auto" },
    { "url": "https://hnrss.org/frontpage", "mode": "rss" }
  ],
  "maxArticlesPerSource": 25,
  "includeUnchanged": false
}

See ./examples/input.json.

Output

Each dataset item describes one article change:

Field	Description
status	new, updated, or unchanged
title	Article title
publishDate	Publish date in ISO format when available
author	Author name when available
url	Canonical article URL
sourceUrl	Monitored source URL from input

A JSON run report is also stored under the default key-value store record OUTPUT:

{
  "checkedAt": "2026-07-05T12:00:00.000Z",
  "sourcesChecked": 2,
  "articlesFound": 40,
  "newCount": 3,
  "updatedCount": 1,
  "unchangedCount": 0,
  "errors": []
}

See ./examples/output.json.

How it works

1. Load previous monitor state from the KV store key MONITOR_STATE.
2. For each source:
   • rss mode fetches the feed directly
   • auto mode tries feed discovery, then RSS parsing, then blog index crawling
   • blog mode crawls the index/listing page
3. Build a fingerprint from title, publishDate, and author for each article.
4. Compare against stored fingerprints to determine new, updated, or unchanged.
5. Push changed articles to the dataset and update KV state for the next run.

Scheduling

Run this Actor on a schedule (e.g. hourly or daily). Only new and updated articles are emitted by default, making it ideal for alerting pipelines and content intelligence workflows.

Local development

npm install
npm test
npm run lint
npm run build
apify run

Use storage/key_value_stores/default/INPUT.json for local input.

Deploy

apify login
apify push

Pricing

Configure pay-per-result pricing in Apify Console using the synthetic event apify-default-dataset-item at $0.002 per result.

Notes

• Feed and HTML structures vary by site; use mode: "rss" when you know the feed URL.
• First run marks all discovered articles as new.
• State persists in the Actor's default key-value store between runs of the same Actor task/schedule.