Pricing

from $0.35 / 1,000 output items

Merge & Deduplicate Datasets - Reliable, Any Size

Merge any number of Apify datasets into one and remove duplicates by the fields you choose (url, id, email...). Keep unique items or only the duplicates, pick output fields, tag each item with its source. Streams data, so millions of items work. Pick datasets from your storage: no permission errors.

Pricing

from $0.35 / 1,000 output items

Rating

0.0

(0)

Developer

Ponytail Data

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Merge, Dedup & Transform Datasets - Limited permissions

lukaskrivka/dedup-datasets-limited-permissions

Limited permission version of the popular Actor. The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

349

Dataset Deduplicator

automation-lab/dataset-dedup

Merge and deduplicate Apify datasets by any field combination. Remove duplicates, keep first or last occurrence. Case-insensitive matching, whitespace trimming. Pay per 1K items processed.

Automation Lab

5.0

(1)

Merge, Filter, Deduplicate & Transform Datasets — CSV, Excel

tidydata/merge-dedupe-datasets

Filter, merge and deduplicate Apify datasets, CSV, Excel, JSON and Google Sheets: keep only rows matching your conditions, remove or merge duplicates (smart URL/email/phone matching), sort, add computed columns, split lists into rows, keep only new rows and export CSV, Excel or JSON.

tidydata

Dataset Validity Checker

equidem/Dataset-Validity-Checker

Automatically checks, whether default datasets created by runs of an actor differ too much from the previously encountered ones, allowing it to warn you about web scraping problems caused by, e.g., a website layout changing, or other significant changes in the resulting data.

Matěj Sochor

Monitoring Checker Duplication

apify/monitoring-checker-duplication

The Monitoring checker duplication is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Apify

5.0

(5)

Data Deduplicator

parsebird/dataset-deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

ParseBird

Merge Datasets & Remove Duplicates: Combine Runs, CSVs, Sheets

nerolabs/dataset-merge-dedupe

Returns one de-duplicated list from many Apify datasets, CSV, Excel or JSON files and Google Sheets, with blanks filled from the copies and each row's source labelled. Inputs: datasetIds, fileUrls, dedupKeys. Charged per merged row written ($0.001). Agent-ready via x402 and MCP.

Adam Pearce

Line Deduplicator, Sorter & Shuffler

hipersoft/line-dedup-sort

Clean big text and list files: remove duplicate lines, sort alphabetically or numerically, reverse, shuffle and trim blanks — handles lists too large for a desktop editor.

hiper soft

JSON Dataset Cleaner and Deduplicator

rodrgds/dataset-cleaner

Clean JSON datasets, remove empty rows, deduplicate by any field, validate emails, and prepare scraper output for CRMs, analysis, or AI workflows.

Rodrigo Dias

CSV Deduper Normalizer

junipr/csv-deduper-normalizer

Deduplicate and normalize CSV-style rows. Clean whitespace, casing, domains, URLs, and emails, then output kept and duplicate rows plus clean CSV/JSON files.