Delta-Sync Sentinel

Stateful Data Observability & Ingestion Infrastructure.

Delta-Sync Sentinel is not a scraper — it's a transactional-outbox sync engine. It fetches sources, splits them into content-addressed chunks, and commits them as immutable manifests under a monotonic cursor. Every design decision below serves one property: re-execution is harmless, and manifest-commit is the single source of truth. That's what makes it safe to sit underneath an autonomous agent or a production ingestion pipeline, rather than just a one-off script.

Consumer contract (read this first)

Downstream consumers MUST upsert on chunk_id. Do not append.

• Progress is a cursor, not a clock. Track your position with the integer cursor (a monotonic counter), never with committedAt. The timestamp is informational only.
• Trust the commit boundary. Only process chunks belonging to a manifest whose cursor <= last_cursor (see the health endpoint). Dataset rows with a cursor above the committed head are orphans from an in-flight or dead run and must be ignored.
• Chunks are content-addressed. chunk_id = sha256(source_url + section_path + content_hash). The same input always produces the same id, so upserting is idempotent and replays are free.

Agentic Integration Contract

This actor is designed to be driven by an autonomous agent, not just a human clicking "Start." An agent needs three things to do that safely: how to trigger a sync, how to recover from a gap, and how to tell if it's safe to trust what it's reading. The system prompt below covers all three — copy it directly into an agent's instructions.

You control the "Delta-Sync Sentinel" Apify actor. Follow this contract exactly:

1. To ingest new/changed sources, call the actor with:
   { "op": "sync", "sources": [...urls], "maxCostUsd": "<your ceiling>" }

2. Before trusting any dataset row, check the actor's health endpoint
   (GET /health in Standby mode, or read the run's OUTPUT key-value entry
   after an op:"health" call). Only trust rows where row.cursor <= last_cursor.
   If status is "recovering", wait and re-check — do not read the dataset yet.

3. Never track position by timestamp. Persist only the integer `cursor`.
   If you suspect you missed a commit (a gap between your last known cursor
   and the current last_cursor), call:
   { "op": "reconcile", "fromCursor": <your_last_cursor + 1> }
   This re-emits every manifest you missed, using the same chunk_ids as the
   original run. It is always safe to call this — including redundantly.

4. Always upsert on `chunk_id`. Never append blindly and never assume a
   dataset row is new just because you just saw it — the same chunk_id
   appearing twice means "this is still the same fact," not "this changed."

5. Always set maxCostUsd. The actor self-terminates before overspending;
   it will not silently exceed your ceiling.

Production Guarantees

Fencing tokens. Every run that intends to commit must hold the lease { run_id, epoch, heartbeat }. The epoch is a fencing token: it increments by exactly 1 every time a lease is stolen from a zombie holder, and every commit re-reads the lease and aborts unless (epoch, run_id) still match. This is what makes a hung run safe — if Run A stalls, Run B steals the lease and bumps the epoch, and Run A's late write is rejected rather than silently corrupting state.

Deterministic IDs. chunk_id = sha256(source_url + section_path + content_hash), joined on a null-byte delimiter specifically so that no combination of URL/path can collide across a field boundary. Same input, same id, every time — across retries, across reconciles, across redeploys.

Upsert-only contract. Nothing in this system is designed to be appended to. A dataset row is a fact keyed by chunk_id; seeing it again is confirmation, not a new event. This is what makes replay (via reconcile) free instead of dangerous.

Architecture

1. Stage (outbox). Each chunk is written to the key-value store only, keyed by its deterministic id. Nothing hits the output dataset yet, so a crash mid-run leaves only dead-weight orphans — never a partial commit.
2. Commit (transaction). After all chunks are staged, commit():
   • passes the fencing check,
   • allocates the next cursor,
   • writes the manifest { manifest_id, chunk_ids, cursor, committed_at },
   • emits the referenced chunks to the dataset,
   • advances CURSOR_HEAD — this HEAD advance is the atomic commit point.
3. Reconcile. { "op": "reconcile", "fromCursor": N } re-emits every committed manifest with cursor >= N, reconstructing each chunk from the KV store with its original deterministic id.

Honest trade-off — TOCTOU (Time-of-Check to Time-of-Use). The Apify key-value store has no compare-and-swap, so lease acquisition is a plain read-modify-write and has an inherent TOCTOU window: two runs could both observe a stale lease and both attempt to steal it at nearly the same moment. We do not pretend otherwise. What actually provides safety is the fencing token, checked at the commit boundary rather than the acquisition boundary — the last writer to steal the lease wins the run_id slot, and whichever run loses that race gets rejected the moment it tries to commit, before it can advance CURSOR_HEAD or affect a consumer. Acquisition can race; the committed state cannot. True CAS-level single-writer acquisition would require a coordination store that supports it (e.g., a database with real optimistic locking) — we're naming that limitation here on purpose, because a system that's honest about where its safety actually comes from is easier to trust than one that claims perfection.

Lifecycle example: No-Op → Manifest Commit

A consumer polling the health endpoint sees nothing change until a sync actually commits:

// Poll 1 — no new data since the consumer's last read. No-op from the
// consumer's perspective: last_cursor is identical to what it saw before.
{ "status": "ok", "staleness_s": 340, "last_cursor": 5 }

// A "sync" run executes in between polls, stages 3 chunks, and commits:
{
  "manifestId": "9697071a4e2c...",
  "cursor": 6,
  "chunkIds": ["b3c3e00b...", "aa740144...", "e9fb4a91..."],
  "committedAt": "2026-07-12T09:14:02.331Z",
  "runId": "run-abc123",
  "epoch": 1
}

// Poll 2 — cursor advanced. The consumer now knows there is exactly one
// new manifest (cursor 6) to upsert, and nothing before it needs re-checking.
{ "status": "ok", "staleness_s": 4, "last_cursor": 6 }

Health endpoint

In Standby mode the actor serves a free-access endpoint at GET /health:

{ "status": "ok" | "recovering", "staleness_s": 42, "last_cursor": 7 }

• staleness_s: seconds since the last committed manifest (-1 if none yet).
• status: recovering when nothing has committed yet, or a lease is held with a stale heartbeat.

Input

Field	Type	Notes
op	sync | reconcile | health	Default sync.
sources	string[]	http(s) URLs to fetch and chunk (for sync).
fromCursor	integer	Re-emit manifests with cursor >= this (for reconcile).
maxCostUsd	string	Required. Hard cost ceiling; the run aborts before staging a chunk that would exceed it. String because the Apify form has no float type.
costPerChunkUsd	string	Optional override for the assumed per-chunk cost used to project against maxCostUsd. Defaults to 0.0005.
baseCostUsd	string	Optional override for the assumed fixed run cost used to project against maxCostUsd. Defaults to 0.01.
chunkSizeChars	integer	Target chunk size when a source has no headings.
heartbeatIntervalSecs	integer	Lease heartbeat cadence; keep well under the 90s zombie threshold.

Output (per committed chunk)

{
  "cursor": 1,
  "manifestId": "9697071a…",
  "chunkId": "b3c3e00b…",
  "sourceUrl": "https://…/README.md",
  "sectionPath": "0:Installation",
  "contentHash": "…",
  "committed": true,
  "runId": "…"
}

Modules

• hasher.ts — DeterministicHasher (content-addressed ids, null-byte-delimited).
• lease-manager.ts — LeaseManager (acquire, heartbeat, zombie-steal, assertFenced).
• manifest-manager.ts — ManifestManager (stage, commit, reconcile).
• extractor.ts — deterministic fetch + sectioning.
• health.ts — Standby health server.
• retry.ts — structured backoff that never retries control-flow errors.
• main.ts — validation, cost ceiling, op dispatch.

Local development

npm install
npm run typecheck
npm run harness   # in-process invariant tests (deterministic ids, monotonic cursor, reconcile, fencing)
npm run build && apify run   # real end-to-end sync