JSON Catalog Deduplicator

Deduplicate, Merge & Transform Datasets

datacach/deduplicate-datasets

Merge multiple datasets, deduplicate items by a combination of fields, and apply custom transforms — powered by Polars.

Merge, Dedup & Transform Datasets

lukaskrivka/dedup-datasets

The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Dataset Deduplicator & Cleaner

eszetael_lab/dataset-deduplicator-cleaner

Returns your records with duplicates removed and values normalized — same fields as the input, nothing added or renamed. Exact, normalized or fuzzy matching on keys you choose. A data quality report lands under QUALITY_REPORT. Takes a dataset ID or inline JSON, so it chains after any scraper.

Dataset Cleaner & Exporter

nerolabs/dataset-cleaner-exporter

Returns a deduplicated, flattened and cleaned copy of any Apify dataset, file or Google Sheet by URL, or JSON array, plus CSV and Excel files, named-dataset append and webhook delivery. Inputs: source, dedup keys, flatten, cleaning options. Agent-ready: pay per event (x402, MCP), per cleaned record.

Unique handle keys from a public JSON catalog. Try is ColourPop — distinct from v1 Allbirds. $0.002 / unique row.

Diffs two public JSON catalogs. Try is Allbirds vs ColourPop products.json. Distinct pair from v1. $2 / 1,000.

Field-level audit of a public JSON array. Try is Allbirds products.json. Not an Apify platform scanner. $2 / 1,000.

Typed rows from a public JSON catalog. Try is ColourPop — distinct from v1 Allbirds. $0.002 / row.

Extract tables from public HTML or text-layer PDFs. Try is a live Wikipedia GDP table. No OCR. $0.01 / row.

OSV.dev advisories for npm lodash. Same class as GitHub Advisories — Try package is lodash. $0.01.