# Changelog of Dataset Change Tracker (`quietworkslab/dataset-change-tracker`) Actor

- **URL**: https://apify.com/quietworkslab/dataset-change-tracker/changelog.md
- **Full Actor documentation**: https://apify.com/quietworkslab/dataset-change-tracker.md

## Changelog

### 0.1 (published on Apify Store on 2026-11-01)

- Next build (2026-11-01): both datasets are read until a page comes back empty or shorter than asked, instead of stopping at the item count the API reports. Right after items are pushed (for example when an integration starts this Actor the moment a scraper finishes) that count can be behind, even 0, and earlier builds then silently left out the items after it. A last page that is exactly full costs one extra (empty) request.
- Build 0.1.2: if the dataset is passed only in an integration's `payload` (which this Actor cannot read with limited permissions), the run stops before charging and says to set `datasetId` to `{{resource.defaultDatasetId}}`.
- Build 0.1.2: the memory check also runs after every page of the previous dataset, never plans with more heap than the run's memory, and Node starts with a heap sized from the run's memory (3/4 of it) and a small young generation.
  - Because the heap is now checked against its real limit, the check accepts fewer items than in build 0.1.1 (items of about 350 bytes): with 256 MB about 90,000 instead of about 130,000; with 1024 MB about 500,000 instead of about 980,000 (0.1.1 checked only the run's memory; that many items were never tested).
- Build 0.1.4: between garbage collections the memory check uses the current rss, so memory outside the heap that grew since the last collection (for example the loaded fingerprint) is counted before charging.
- Build 0.1.3: pages grow at most twice as large as the page before (100, 200, 400, ... 5,000 items), so items that become large after a first page of small ones no longer make one huge page.
- Build 0.1.3: a run restarted by Apify (for example moved to another server) that stops at the memory check keeps a PARTIAL summary with the items it compared and charged before the restart, instead of saying that nothing was charged.
- Build 0.1.3: near the memory limit, garbage is collected only after the heap has grown by a tenth of its limit since the last collection, not on every page.
- Build 0.1.3: the README shows the prices.
- First version: compare two datasets, or compare with the hash-only fingerprint of the last run.
- Key fields, ignored fields (with dotted paths), optional order-insensitive lists, choice of change types.
- Changed items list the changed fields and their previous values.
- Pay-per-event: one `item-compared` event per item of the current dataset; stops cleanly at the maximum cost per run.
- Checks before any charge: key fields present in the first items, two different datasets, enough memory, readable fingerprint.
- Each page is charged right after it is saved; only a page whose charge reaches the maximum cost waits until the final summary is saved. An interrupted run continues without charging an item twice.
- Memory check before charging counts both the JavaScript heap and memory outside it, against the run memory.
- Platform migrations, aborts and the run timeout (30 s before it) are handled by the Actor itself, also outside the page loop, so the summary is always written. A stop at the maximum cost keeps its message when Apify aborts the run afterwards.
- The fingerprint is committed in two steps, so a restart right after saving it does not compare with the new fingerprint. Records left by an interrupted save are deleted when the run restarts or the next fingerprint is saved.
- Hidden fields (`#...`) are not compared; pages are sized by bytes; compact in-memory sets for keys and fingerprints.
- README only (2026-11-01): links to the example tasks (ready-made inputs with their own pages) and to the other Actors of Quiet Works Lab.