Merge Apify datasets from many runs, plus CSV, JSON or Excel files, into one table and remove duplicates by any field. Ignore case, spaces and URL tracking; keep the first, last or most complete row; get only rows new since earlier runs, or keep a master list. Runs with limited permissions.
Versions follow MAJOR.MINOR.PATCH (src/version.py); Apify shows MAJOR.MINOR from .actor/actor.json.
Every run logs its version and records it in the RUN_STATS key-value record.
0.1.0 (2026-11-04)
First release.
Merges up to 200 Apify datasets and CSV/TSV, JSON, JSON Lines or Excel files into one dataset, under limited
permissions (the datasets are declared READ in the input's picker). In an integration, set
"datasetIds": ["{{resource.defaultDatasetId}}"]: a real integration run on Apify got HTTP 403 on the same dataset read
from the payload (2026-11-04), so an empty field ends with that instruction as the status.
Dedupes by any fields, nested paths or the whole row, optionally ignoring spaces, case, accents, URL details and
everything but digits. Keeps the first, last, most complete or a combined row.
Across runs: only rows not returned earlier (64-bit key fingerprints in a named key-value store), and a master dataset
that grows without duplicates.
An unreadable source is reported and the others carry on. If no row has the dedupe fields, the run stops before writing.
File links reach logs, stats and rows without their query string or password; the CSV guards formula-like cells.
SUMMARY record with counts overall and per source; optional mergeSource field; optional CSV or JSONL file.
Charged per row written (apify-default-dataset-item); duplicates and already-returned rows are free. Max rows per run
and the maximum cost per run stop the run.
Sources: the user's own datasets and files they supply, read through the shared guarded client. The example uses the US
Treasury's Fiscal Data API (https://fiscaldata.treasury.gov/api-documentation/ , read 2026-11-04: "offered free, without
restriction").