News Delta API — only new articles from Google News + RSS/Atom

A news API and RSS/Atom monitor that returns only new articles since your last run. Point it at Google News keyword/topic queries and/or public feed URLs, give each monitor a stable monitorId, and every subsequent run delivers canonical, deduped article metadata you have not seen before — ready for an LLM, agent, webhook, or spreadsheet.

Why delta

Pull scrapers and raw feed readers re-send the same headlines every poll. You end up building your own seen-set, URL normalizer, and cross-source dedup. This actor keeps a persistent per-monitorId seen-set in a named key-value store and returns the delta only: title, canonical publisher URL, source domain, published date, and snippet. No publisher full text (copyright + scope). First run is a free baseline (records state, delivers no article rows) unless you set deliverOnFirstRun.

Quickstart (Console)

1. Open the actor → Input.
2. Keep the prefill query artificial intelligence or add your own queries / feeds.
3. Set monitorId (e.g. ai-watch) so later runs share the same seen-set.
4. Start — first run writes a baseline row. Run again with the same monitorId to receive article rows.

Input reference

Field	Type	Default	Notes
queries	string list	prefill ["artificial intelligence"]	Google News search queries; supports site:, quotes, OR, when:7d
feeds	string list	[]	RSS 2.0 / Atom / JSON Feed URLs (http(s), public hosts only)
monitorId	string	"default"	Identity of the persistent seen-set
maxArticlesPerQuery	integer	50 (max 100)	Newest-first cap per Google News query
maxTotalNewArticles	integer	500	Cap on delivered article rows per run
language	string	"en"	Google News hl
country	string	"US"	Google News gl; ceid = country:language
deliverOnFirstRun	boolean	false	If true, first run delivers + charges like a normal delta run

Output contract

Four row types in the default dataset:

article (charged as new-article)

{
    "type": "article",
    "monitorId": "ai-watch",
    "sourceType": "query",
    "query": "artificial intelligence",
    "title": "Example headline",
    "url": "https://publisher.example/story",
    "urlIsGoogleRedirect": false,
    "sourceDomain": "publisher.example",
    "publishedAt": "2026-08-01T12:00:00.000Z",
    "snippet": "Plain-text snippet up to 500 chars",
    "firstSeenAt": "2026-08-01T13:00:00.000Z"
}

baseline (uncharged, first run only)

{
    "type": "baseline",
    "monitorId": "ai-watch",
    "sources": [{ "source": "artificial intelligence", "sourceType": "query", "status": "ok", "itemsFetched": 50, "newItems": 50 }],
    "message": "Baseline recorded — subsequent runs deliver only new articles."
}

summary (uncharged, always last)

{
    "type": "summary",
    "monitorId": "ai-watch",
    "baseline": false,
    "runCount": 2,
    "sources": [{ "source": "artificial intelligence", "sourceType": "query", "status": "ok", "itemsFetched": 50, "newItems": 3 }],
    "totals": { "sourcesPolled": 1, "newArticles": 3, "duplicatesSkipped": 47, "decodeFailures": 0 },
    "chargeLimitReached": false
}

error (uncharged)

{
    "type": "error",
    "error": "no-sources",
    "message": "Provide at least one Google News query in `queries` or one feed URL in `feeds`.",
    "monitorId": "default"
}

Per-source failures (sourceType: "query"|"feed") also emit error rows; the run still succeeds. Other run-level error codes (also uncharged, run still succeeds): no-sources, concurrent-run (another run currently holds this monitorId's lease — nothing polled or charged), state-read-failed / state-write-failed (monitor state store temporarily unavailable — nothing polled or charged).

monitorId + baseline

• Same monitorId → shared seen-set across runs (and schedules).
• First run with a new monitorId (runCount === 0) and deliverOnFirstRun: false: polls all sources (charges query-poll), records every item into seen state, delivers zero article rows and zero new-article charges, pushes one baseline row.
• The baseline is completed per source. A source that failed, or was skipped for budget or time, was never read, so its existing catalogue was never recorded — it is baselined free on a later run instead of being billed as "new". Sources that did complete move to delta billing immediately, so one broken feed never silences the rest of the monitor. While any source is still unbaselined the baseline/summary rows carry baselineIncomplete: true.
• Later runs deliver only identities not in the seen-set (URL key, Google News id, or title hash within 14 days). Items with no title are identified by URL alone, so several untitled items from one feed are never collapsed into a single row.
• An item whose only identifier is an opaque <guid> or JSON Feed id (urn:uuid:…, tag:…, a bare number) carries no link, so it is skipped rather than delivered — and billed — with a URL that cannot be opened.
• Only one run at a time may hold a given monitorId. The lease is a server-side lock, so the second run exits successfully with a concurrent-run error row and charges nothing — two runs can never sell you the same article. A run that crashes releases its monitor within about 8 minutes; a run that is still alive keeps it. Schedule a monitor no more often than its runs take to finish.

Recipes

Schedule every 30 minutes

Apify Schedules → this actor, interval 30 minutes, payload:

{
    "queries": ["artificial intelligence", "site:reuters.com chips"],
    "feeds": ["https://techcrunch.com/feed/"],
    "monitorId": "ai-watch",
    "maxArticlesPerQuery": 50
}

API (run-sync-get-dataset-items)

curl -s "https://api.apify.com/v2/acts/kaz_kakyo~news-delta-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "content-type: application/json" \
  -d '{"queries":["artificial intelligence"],"monitorId":"api-demo"}'

First call baselines; second call returns new articles (plus summary).

MCP / agents

Works with the Apify MCP server — agents can call this actor as a tool with the same JSON input. Use a stable monitorId per watch topic so the tool returns deltas instead of full dumps.

Pricing

Pay-per-event (configure prices in Console):

Event	Meaning
apify-actor-start	Once per successful validated run (~$0.005)
query-poll	Per source (query or feed) successfully polled — 200 and 304 both count (~$1 / 1,000)
new-article	Per delivered article row (~$1 / 1,000)

A quiet monitor that polls one source and finds nothing new costs roughly one start + one poll (~$0.006/poll at those list prices). Failed fetches and SSRF-blocked feeds are not charged as polls. Invalid empty input charges nothing.

Honest limits

• Google News returns up to ~100 items per query; we cap with maxArticlesPerQuery.
• Canonical-URL decode uses a Google internal batchexecute path that can drift; on failure we still deliver the news.google.com redirect link with urlIsGoogleRedirect: true.
• No article full text — metadata and snippets only.
• Fuzzy title dedup uses a 14-day window (generic headlines can collide).
• Seen-set retention per monitor: 60 days, capped at 30,000 URL + 30,000 Google News + 10,000 title identities (oldest evicted first). An article resurfacing after falling out of that window is treated as new again.
• Each feed has a 20-second total fetch budget (redirects included) and a 5 MB body cap.
• Feeds must be publicly reachable; private/internal hosts (localhost, RFC1918, link-local, cloud metadata) are blocked by the SSRF guard. URLs with embedded user:pass@ credentials are rejected — use a token query parameter if your feed needs auth.
• Authenticated feed URLs are never echoed back in full. Wherever a feed URL appears in output (article rows, error rows, summaries, logs) its query-parameter values are replaced with *** — parameter names, host and path are kept, so https://host/feed.xml?edition=uk&api_key=SECRET is reported as https://host/feed.xml?edition=***&api_key=***. Credential-looking path segments (/reset/SECRET/…) and any user:pass@ are masked the same way.

Local development

npm install
node src/unit-check.js
apify run --purge -i '{"queries":["artificial intelligence"],"monitorId":"local-test"}'