Remove Duplicates: Deduplicate & Merge Datasets by Field

Merge Datasets & Remove Duplicates — Deduplicate, Filter, Excel

tidydata/merge-dedupe-datasets

Merge datasets and remove duplicates in one call: combine Apify datasets, CSV, Excel, JSON or Google Sheets, dedupe by any field (smart URL/email/phone matching), filter, sort, keep only new rows, export CSV/Excel/JSON. Works from AI agents via Apify MCP (Claude, ChatGPT), n8n and the API.

Dataset Transformer: Filter, Dedupe, JSON to CSV

humble-echidna/dataset-transform

Clean, deduplicate and transform data: an Apify dataset, pasted JSON or a CSV, JSON, JSONL or Excel XLSX file URL. Filter rows, remove duplicates by any field, sort, rename fields, flatten JSON. Get a dataset and an optional CSV or JSONL file: JSON to CSV, CSV to JSON, Excel to CSV or Excel to JSON.

Merge & Dedup Datasets: Remove Duplicates, Only New Items

charlesmarkette/merge-deduplicate-datasets-only-new-items

Merge any number of Apify datasets and remove duplicates by the fields you choose (or the whole item). Keep the first or last copy, or output only items not seen in earlier runs: ideal after a scheduled scraper. Streams millions of items. Duplicates are free; $0.0019 per 1,000 items output.

Dataset Cleaner & LLM Processor: Filter, Dedupe, AI per Row

automa-flow/dataset-filter-transform

Clean, filter and dedupe Apify Dataset rows, then run an AI prompt on each row with no API key of your own. $1 per 1,000 rows kept, plus $1 per 1,000 AI rows, plus model usage on your Apify account.

Dataset Aggregate, Group By & Pivot

nerolabs/dataset-aggregate-pivot

Returns GROUP BY and pivot tables for any Apify dataset, file or Google Sheet by URL, or JSON array: count, sum, average, min, max, median per group, date buckets, pivot columns. Exports CSV/Excel, appends to a named dataset, posts to a webhook. Agent-ready: pay per event (x402, MCP), per input row.

Python Code Runner - pip, pandas & Dataset In/Out

automa-flow/python-code-runner

Run your Python main() on rows from a Dataset, JSON or n8n in one API call, or a Python script on a schedule. pip packages from a requirements field, pandas and openpyxl preinstalled, encrypted secrets as environment variables, rows back with per-item status.

Data Deduplicator

parsebird/dataset-deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

Dataset Toolbox

cyberfly/dataset-toolbox

Perform common actions on datasets - merge, unify, validate, transform, order fields etc.

Merge Apify datasets, CSV, JSON, JSONL or Excel files and pasted rows into one clean dataset: remove duplicates by any fields (case and whitespace normalised), keep only items new since your last run, left/inner join on a key, flatten nested JSON. $0.05 per 1,000 rows out.

Merge & Deduplicate Datasets - Reliable, Any Size

geese_chief/dataset-merge-dedup

Merge any number of Apify datasets into one and remove duplicates by the fields you choose (url, id, email...). Keep unique items or only the duplicates, pick output fields, tag each item with its source. Streams data, so millions of items work. Pick datasets from your storage: no permission errors.