Pricing

from $0.20 / 1,000 rows

Try for free

Go to Apify Store

Merge, Dedupe & Transform Datasets

Try for free

Combine several Apify datasets into one clean dataset: remove duplicates by any field, keep first or newest copy, flatten nested data, pick, drop or rename columns. Pay only for rows written.

Pricing

from $0.20 / 1,000 rows

Rating

0.0

(0)

Developer

BrightLedger

Actor stats

Bookmarked

Total users

Monthly active users

10 days ago

Last modified

Common uses

Combine daily or weekly scrapes into one list with no repeats (dedupe by url).
Merge lead lists from different tools and remove duplicate companies (dedupe by domain) or people (dedupe by email).
Keep the freshest data: choose Last seen to keep the newest copy of each record in its original position.
Prepare for export: flatten nested fields and keep only the columns you need before sending to CSV, Excel, or Google Sheets.

How matching works

Duplicates are matched on the fields you list, across all datasets, in the order you give them.
By default matching ignores case and extra spaces (Acme.com = acme.com ).
Rows where all key fields are empty are kept as they are, never merged into one.
Nested fields can be used as keys with dots, e.g. socials.linkedin.

Input tips

Run outputs are unnamed datasets, so they may not show in the dataset picker. Switch the input to JSON and paste their IDs (open a run, then the Storage tab, then copy the Dataset ID):

{
  "datasetIds": ["FIRST_DATASET_ID", "SECOND_DATASET_ID"],
  "dedupeFields": ["domain"],
  "keep": "first"
}

Output

The combined rows in this run's dataset, with an optional _sourceDatasetId column.
A summary record: items read per dataset, duplicates removed, rows written.

Pricing

You pay per row written to the output. Removed duplicates are free.

Works well with

Company Website to AI Company Profile (brightledger/company-website-profiler): merge profiles from several runs, dedupe by domain.
Google Sheets Export (brightledger/google-sheets-export): send the clean result to your sheet.

Merge, Dedup & Transform Datasets

lukaskrivka/dedup-datasets

The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

5.4K

5.0

(2)

Dataset Filter & Transform

nerolabs/dataset-filter-transform

Returns rows of any Apify dataset, CSV/Excel/JSON file URL or Google Sheet that pass filter rules (25 operators, AND/OR, date ranges that fix a scraper's ignored date filter) after transforms (rename, cast, compute, regex), then sort, dedupe, export, webhook. Agent-ready: pay per event (x402, MCP).

Adam Pearce

177

Merge, Dedup & Transform Datasets - Limited permissions

lukaskrivka/dedup-datasets-limited-permissions

Limited permission version of the popular Actor. The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

601

Dataset Join & Merge (VLOOKUP for Datasets)

nerolabs/dataset-join-merge

Returns two Apify datasets, files or Google Sheets by URL, or JSON arrays joined on key fields like SQL or VLOOKUP: left, inner, right, full and anti joins plus union, with CSV/Excel export, named-dataset append and webhook delivery. Agent-ready: pay per event (x402, MCP), per output row.

Adam Pearce

Merge Datasets & Remove Duplicates — Deduplicate, Filter, Excel

tidydata/merge-dedupe-datasets

Merge datasets and remove duplicates in one call: combine Apify datasets, CSV, Excel, JSON or Google Sheets, dedupe by any field (smart URL/email/phone matching), filter, sort, keep only new rows, export CSV/Excel/JSON. Works from AI agents via Apify MCP (Claude, ChatGPT), n8n and the API.

tidydata

Dataset Transformer: Filter, Dedupe, JSON to CSV

humble-echidna/dataset-transform

Clean, deduplicate and transform data: an Apify dataset, pasted JSON or a CSV, JSON, JSONL or Excel XLSX file URL. Filter rows, remove duplicates by any field, sort, rename fields, flatten JSON. Get a dataset and an optional CSV or JSONL file: JSON to CSV, CSV to JSON, Excel to CSV or Excel to JSON.

Michael Costa

Deduplicate, Merge & Transform Datasets

datacach/deduplicate-datasets

Merge multiple datasets, deduplicate items by a combination of fields, and apply custom transforms — powered by Polars.

DataCach

LLM Dataset Processor

dusan.vystrcil/llm-dataset-processor

Allows you to process output of other actors or stored dataset with single LLM prompt. It's useful if you need to enrich data, summarize content, extract specific information, or manipulate data in a structured way using AI.

Dušan Vystrčil

158

Fast Dataset Cleaner & CSV Formatter

motivational_nickel/dataset-cleaner-and-formatter

Fast dataset cleaning for CSV and JSON files. Automatically removes duplicates, trims whitespace, fixes capitalization, and normalizes fields. Works with Apify datasets or uploaded files and prepares data for analytics, CRM imports, and automation pipelines.

Leoncio Coronado Jr

Dataset Cleaner & LLM Processor: Filter, Dedupe, AI per Row

automa-flow/dataset-filter-transform

Clean, filter and dedupe Apify Dataset rows, then run an AI prompt on each row with no API key of your own. $1 per 1,000 rows kept, plus $1 per 1,000 AI rows, plus model usage on your Apify account.