JSON Catalog Deduplicator v2

Data Deduplicator

parsebird/dataset-deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

Deduplicate, Merge & Transform Datasets

datacach/deduplicate-datasets

Merge multiple datasets, deduplicate items by a combination of fields, and apply custom transforms — powered by Polars.

Diffs two public JSON catalogs. Try is Allbirds vs ColourPop products.json. Distinct pair from v1. $2 / 1,000.

Typed rows from a public JSON catalog. Try is ColourPop — distinct from v1 Allbirds. $0.002 / row.

Field-level audit of a public JSON array. Try is Allbirds products.json. Not an Apify platform scanner. $2 / 1,000.

🧽 Dataset Deduplicator - Clean Any Scraped Dataset

that_red_bird/dataset-deduplicator

🧽 Upload any dataset — scraped rows, a RAG corpus, an export — and get back exact AND near-duplicates removed, plus a per-field data quality report. ✅ MinHash + LSH banding finds near-duplicate text at scale without O(n²) pairwise comparison.

Validate Dataset(s) with JSON Schema

jaroslavhejlek/validate-dataset-with-json-schema

This Actor validates items in one or more datasets against a provided JSON Schema. Use it if you planning to add a dataset validation schema to your actor and you want test it.

JSON Catalog Deduplicator

zentrafoundry/dataset-deduplicator

Unique handle keys from a public JSON catalog (Try is Allbirds). Not an Apify-platform merge. $0.002 / unique row.

JSON Dataset Cleaner and Deduplicator

rodrgds/dataset-cleaner

Clean JSON datasets, remove empty rows, deduplicate by any field, validate emails, and prepare scraper output for CRMs, analysis, or AI workflows.

OSV.dev advisories for npm lodash. Same class as GitHub Advisories — Try package is lodash. $0.01.