🧹 Merge, Deduplicate & Export Datasets — CSV, Excel, JSON

Combine several Apify datasets (or CSV/JSON files) into one clean dataset: remove duplicates, filter rows, rename or drop columns, keep only the rows you haven't seen in earlier runs, and export CSV, Excel or JSON with a download link. Built for scheduled scrapers that produce overlapping results.

• ✅ Pick datasets from a list: the Apify selector gives this Actor exactly the permission it needs (no API tokens to paste)
• ✅ Merge any number of datasets and public CSV/JSON files in one run
• ✅ Deduplicate by any field(s) (url, id, or a combination), keep the first or the last (most recent) row
• ✅ "Only new rows": remembers what earlier runs saw, so each scheduled run outputs just the new items
• ✅ Filters in plain text: price < 50, country = ES, title contains phone, email exists
• ✅ Rename / drop columns, flatten nested JSON (price.value) so it fits a spreadsheet
• ✅ CSV (Excel-friendly UTF-8), XLSX and JSON files with download links; CSV/Excel cells that start with =, +, - or @ are neutralised so scraped text can't run as a formula
• ✅ Clear error messages that tell you what to fix
• ✅ Free demo if you run it with an empty input

How to use it

1. Datasets to merge: pick them from your storages (or paste IDs). Optionally add CSV/JSON links.
2. Remove duplicates by: type the field(s), e.g. url. Choose first or last.
3. Optional: filters, columns to drop/rename, Only new rows, and the file formats you want.
4. Run. The result is in the dataset; files are in the Output tab.

Run it automatically after another Actor

Add it as an integration of your scraper and set Datasets to merge to {{resource.defaultDatasetId}}. Turn on Only new rows and every scheduled run outputs just the new items.

Filters

One rule per line, all rules must pass. field operator value

Operator	Example
=, !=	country = ES
>, >=, <, <=	price < 50 (numbers are compared as numbers)
contains, not_contains	title contains phone
exists, not_exists	email exists

Filters and duplicate keys use the original column names; renaming/dropping happens afterwards.

Pricing

• $0.25 per 1,000 rows in the result ($0.00025 each)
• $0.005 per file created (CSV, XLSX or JSON)

Rows removed as duplicates or by filters are free. A 100,000-row dataset reduced to 60,000 unique rows costs $15.

Notes

• Memory for "only new rows" is a named storage in your account (default name merge-dedupe-memory); use a different name per project, or delete the storage to start over.
• Big merges happen in memory: up to ~2,000,000 rows.
• Only your own data is processed; nothing is sent anywhere else.