Dataset Cleaner — Visual Dedup, Merge & Export (No Code)

Clean, dedupe, merge, and export any Apify dataset — or your own CSV/JSON file — from a spreadsheet-style web UI. No JavaScript, no config files, no code. Point it at one or more of your datasets (or just drag a file in), click through checkboxes and dropdowns, and download a clean CSV or XLSX (or a brand-new Apify dataset) in minutes.

If you've ever opened lukaskrivka/dedup-datasets and been asked to write a JSON config with a custom JS deduplication function, this actor is the alternative: everything — column selection, duplicate keys, filters, renames — happens by clicking, in a live preview, with a running cost estimate.

What it does

• Remove duplicates — pick one or more key columns, choose whether to keep the first or last occurrence, and optionally ignore case/whitespace differences.
• Filter rows — contains / equals / empty / not empty / regex, combinable across as many columns as you need.
• Merge datasets — select several datasets and clean them together as one. The moment you check a second one, a clearly labeled merge banner appears showing exactly what's about to be combined, the total row count, and whether the selected datasets share the same columns (with a breakdown of the differences if they don't) — merging is never a surprise.
• Upload your own file — drag and drop a CSV, TSV, JSON, or JSON Lines file straight into the UI; it becomes a dataset you can preview and clean exactly like one already on your account. No need to load it into Apify first.
• Rename, remove, and reorder columns — shape the final table before export.
• Auto-flatten nested data — nested objects become parent.child columns automatically, so messy scraper output turns into a flat, exportable table.
• Live cost preview — see the estimated row count and price update as you configure the pipeline, before you commit to anything. Once pay-per-event pricing is configured for this actor in the Apify Console, the preview automatically switches from suggested defaults to your real, live prices — no code change needed.
• Export anywhere — a new Apify dataset (your source datasets are never modified), plus one-click CSV and XLSX downloads.
• Manual kill switch — a "Kill current session" button in the UI stops the actor run immediately; there's no automatic idle timeout, so it's on you to end the session when you're done.

Getting started

Nothing to configure. Click Start in the Apify Console with the default (empty) input.

The actor boots a small web server inside the run and immediately prints a link — watch the Log tab (or the Output tab) for a banner like this:

╔══════════════════════════════════════════════════════════════════════╗
║  🧹 Dataset Cleaner UI — open this link:                              ║
╚══════════════════════════════════════════════════════════════════════╝
https://<your-run>.runs.apify.net/?token=8f3a1c9b...

Click (or copy-paste) that URL — that's the whole setup. It opens the Dataset Cleaner UI, already signed in to that run.

The link only ever appears once, at the top of the logs, right after the run starts. If you scrolled past it, the exact same link is also saved in the run's Output tab under uiUrl.

A visual walkthrough of the UI

Since the interface itself is the whole product, here's what each part does once you're in:

1. Choose dataset(s) (left panel) — every dataset on your account is listed here (name, item count, created date), with a checkbox next to each. Check one to clean it, or several to merge them first. Right below the list is the upload zone: drag a .csv, .tsv, .json, or .jsonl file onto it (or click to browse), and it's parsed and added to the list as a new dataset within seconds — no separate upload step, no Apify dataset to create by hand first.
2. Preview (top middle) — a paginated, spreadsheet-style table of the selected dataset, 15 rows at a time, with columns auto-detected from the data. When 2+ datasets are checked, a "🔗 Merging N datasets into one" banner appears right above the table listing each dataset and its row count, the combined total, and a column-compatibility check — plus each dataset still gets its own tab here so you can check it individually before processing. The Process button itself relabels to "🔗 Merge & Process N datasets" whenever more than one is selected, so it's never ambiguous that clicking it will combine them.
   • If the datasets have different columns, that's shown as a warning (with an expandable breakdown of which columns are unique to which dataset) and nothing is blocked — merging heterogeneous data is expected to leave some blanks.
   • If they have the exact same columns, the banner asks you to explicitly confirm — "Yes, merge them" or "No" — since combining identical-shaped data means the rows can no longer be traced back to their original dataset. Process stays disabled until you confirm.
3. Clean it up (bottom middle, three tabs):
   • Deduplicate — tick the checkbox, click the columns that define a duplicate, choose to keep the first or last occurrence, and optionally ignore case/whitespace.
   • Filters — add as many contains / equals / is empty / is not empty / regex conditions as you need; all must match (AND) for a row to survive.
   • Columns — rename any column inline, tick "remove" to drop one, or use the ↑/↓ buttons to reorder — this is exactly the column set and order your export will use.
4. Cost preview (right panel) — updates live as you change anything: how many rows will be processed, the estimated row count after filters/dedup, and a full breakdown by billing event with a running total. Type a name for the output dataset, then hit Process.
5. Result — once processing finishes, you get the new dataset's name and row count, a link to open it in the Apify Console, and Download CSV / Download XLSX buttons — each showing exactly what that specific download will cost before you click it.

From there you can clean another dataset in the same session, or click Kill current session in the top bar when you're done.

Input

Nothing to configure — the input is intentionally empty. Everything happens inside the UI once the actor is running (see Getting started above).

Pricing — pay per event

This actor uses Apify's pay-per-event billing. There's no charge just for starting the actor — you pay for what you actually process, export, or keep the UI open for.

Event	What triggers it	Suggested price
rows-processed	Per row: once when the pipeline reads it (after merge, before filters), and again for each row in a CSV or XLSX generated for that dataset	~$0.00015 / row (≈ $0.15 / 1,000 rows)
dataset-export	Once when the cleaned dataset is created, and again each time you download a CSV or XLSX	~$0.05 / export
session-minute	Per minute the UI session stays open	~$0.005 / minute

rows-processed is charged one unit per row, not per 1,000-row batch — the price above is deliberately small (a fraction of a cent) since it's a per-row rate. This matters if you're configuring the price yourself in the Console: entering 0.15 there would charge $0.15 per single row, not per 1,000 rows.

Downloading a CSV or XLSX therefore charges both events: a flat dataset-export fee for generating the file, plus rows-processed scaled to how many rows are in it — serializing a file has a real, row-scaled cost, not just a fixed one. The UI shows the exact charge (not an estimate — the row count is already known at that point) right next to the Download buttons after a run completes. Uploading your own file to create a dataset is free — billing only starts once you click Process or download a result.

The exact prices are configured in this actor's Monetization settings in the Apify Console and may differ from the suggested numbers above. The actor reads its own real, currently-configured prices at runtime (via the Apify SDK's Actor.getChargingManager().getPricingInfo()) and uses them for every cost preview in the UI — so once pricing is set up in the Console, the numbers shown match your actual invoice exactly, with no code change on either side. The suggested prices above are only ever shown as a fallback, before pay-per-event pricing has been configured for the actor (the cost panel says so explicitly in each mode).

There is no automatic idle timeout — click Kill current session in the UI as soon as you're done to stop session-minute charges. A forgotten, still-open tab keeps being charged.

Design decisions (documented, not blocking)

A few product choices were made directly rather than left ambiguous:

• Pipeline order is fixed: merge → filters → deduplication → column renaming/removal/reordering. Filters run before dedup so "duplicate" always means "duplicate among rows that already passed your filters."
• "Rows processed" counts source rows, not output rows — it reflects the compute cost of reading and merging, independent of how many survive filtering/dedup.
• Preview is per-dataset, even when merging: when multiple datasets are selected, the preview shows one dataset at a time via tabs (switchable), while filters/dedup/columns apply to the true merged set at Process time. Simulating a merged, paginated preview across heterogeneous datasets added complexity without matching the spreadsheet-style, 15-rows-per-page framing.
• CSV/XLSX export delegates to Apify's own dataset export API (dataset.downloadItems()) rather than a custom XLSX writer — it's what the platform's own "Export" button uses, so formatting always matches, and it avoids re-implementing Excel serialization.
• Deep arrays (arrays of objects, or mixed-type arrays) are flattened to a single JSON-string column rather than spread across multiple columns, since the latter makes the column set unstable across rows with different array lengths. Arrays of plain values (strings/numbers) are joined with ; .
• The row-count estimate is sampled, not exact, once a dataset is larger than roughly 20,000 rows: the actor scans a bounded prefix, measures how many rows survive filters/dedup, and extrapolates. This keeps the "live" cost preview fast on huge datasets. The actual Process run always processes every row exactly.
• The preview is capped at a fixed sample (500 rows), regardless of how many pages you click through, and this is enforced server-side, not just hidden in the UI. The preview exists so you can see your columns and try out filters/renames before paying for anything — it isn't a way to page through and copy out an entire dataset for free. Only Process (charges rows-processed) and the CSV/XLSX downloads (charge both dataset-export and rows-processed) can ever produce the full, complete result. Making the preview harder to select/copy in the browser (e.g. rendering it as an image) was considered and rejected: it's trivially defeated by reading the same JSON straight out of the browser's network tab, so it would add friction for legitimate users without closing anything a determined user couldn't route around in seconds. Capping what the server will ever return for a preview closes the gap for real, everywhere, instead of only in one UI.
• No automatic idle timeout. An earlier version auto-closed the session after N minutes of inactivity; this was removed in favor of a manual Kill current session button. The trade-off is explicit: a forgotten tab now keeps charging session-minute indefinitely instead of self-limiting — ending the session is the user's responsibility, not the actor's.
• rows-processed charges one unit per row, not per 1,000-row batch. An earlier version batched charges into 1,000-row units to make the price look like a round "$0.15 per 1,000 rows." When pricing was configured in the Apify Console using a per-row price ($0.00015, matching that same target rate), the two didn't line up — the code would have charged 1,000× less than intended. The code now always charges per row, matching how the Console's own event pricing (and its title, "Rows processed" / description "Cost per Modified Row") is naturally structured; the per-row price is just the equivalent rate expressed per unit instead of per 1,000.
• No input fields at all. An earlier version had an optional "pre-select a dataset" field; it was removed since picking (or uploading) a dataset inside the UI is just as fast and keeps the Console's input form from implying there's anything to configure before you can start.
• Uploaded files are parsed in-memory and pushed straight into a new Apify dataset rather than processed as a separate "local file" concept throughout the codebase. This means every downstream feature (preview, merge, filters, dedup, export) works on an upload identically to any other dataset, with zero special-casing. The trade-off is a practical size limit: uploads are capped at 20 MB / 200,000 rows — comfortably enough for "clean this CSV I exported from a spreadsheet," but not intended for scraper-scale data (which should already be an Apify dataset).
• CSV values are kept as raw strings — never auto-coerced to numbers or booleans. A leading-zero postal code or a phone number shouldn't silently become a number just because it looks like one; if you need a numeric filter, the "equals"/"contains" comparisons already work against the string form.
• CSV delimiter is auto-detected (comma, semicolon, or tab) by sampling the header row, since semicolon-delimited exports are the default for many regional spreadsheet locales.
• Checking a second dataset shows a merge banner immediately, rather than silently combining datasets at Process time with no prior indication. An earlier version relied on per-dataset preview tabs alone to imply a merge was happening, which wasn't obvious enough — selecting multiple datasets is now impossible to miss (a titled banner, per-dataset row counts, a combined total, and a column-compatibility check), and the Process button itself relabels to "Merge & Process N datasets" as an extra, unambiguous confirmation.
• Mismatched columns merge automatically (with a warning); identical columns require an explicit Yes/No. These are treated differently on purpose: if the selected datasets have different columns, that's obviously heterogeneous data and blanks are an expected, low-stakes outcome — a warning is enough, nothing is blocked. If they have the exact same columns, merging is silent and total: rows lose any trace of which dataset they came from. That's confirmed explicitly ("Yes, merge them" / "No") rather than just noted in passing, and Process stays disabled until you do. Any change to the selection resets this confirmation, since it applies to one specific set of datasets, not to "whatever happens to be checked."

Security

• A random session token is generated for each run and is required on every UI/API request. It is never the same as your Apify API token, and your Apify API token never reaches the browser or appears in any URL, log, or response — all Apify API calls happen server-side, inside the running actor.
• Source datasets are read-only from this actor's point of view. Every "Process" run writes to a new dataset; nothing you already have is ever overwritten or deleted. The same applies to uploads: a file you drag in becomes a new dataset, it never touches or replaces anything else in your account.
• This actor requires Full permissions. Apify's default "Limited permissions" run token can only see the actor's own storages — it cannot list or read datasets elsewhere in your account, which is the entire point of this tool. Set Actor permissions → Full in this actor's Settings tab in the Apify Console, and approve the one-time confirmation prompt Apify shows the first time a full-permission actor runs. If you see Insufficient permissions for datasets in the logs or an empty dataset list in the UI, this setting is the fix.

FAQ

Does this modify my original dataset? No. Every cleaning run writes to a brand-new dataset. Your source data is only ever read.

Can I merge datasets with different columns? Yes. Columns are auto-detected per dataset; when you merge, the union of columns across your merged rows is preserved (rows missing a given column simply have it blank).

Can I clean a file from my computer instead of an existing Apify dataset? Yes — drag and drop it onto the upload zone in the dataset panel (or click it to browse), and it becomes a new dataset you can preview, merge, and clean like any other. Supported formats: CSV, TSV, JSON (an array of objects, or one object), and JSON Lines (.jsonl/.ndjson, one JSON object per line). There's a 20 MB / 200,000-row limit per upload — for anything larger, load it into an Apify dataset first (via another actor or the API) and select it from the list instead.

What happens to nested/JSON fields, like address: { city, zip }? They're automatically flattened into address.city, address.zip, etc., so they show up as normal spreadsheet columns.

How is deduplication actually computed on huge datasets? The actor streams rows in batches (never loading a full dataset into memory) and tracks duplicate keys, not full row content, so memory use stays proportional to the number of unique keys, not the number of rows.

How do I stop being charged session-minute? Click Kill current session in the UI as soon as you're done. There is no automatic timeout — a forgotten, still-open tab keeps the session (and the charge) running.

Can I use the REST API instead of the UI? The UI is backed by a small JSON API described in the actor's Web Server tab (/api/datasets, /api/upload, /api/estimate, /api/process, /api/jobs/:id, /api/datasets/:id/export.csv|xlsx), all guarded by the same session token shown in the run logs/Output.

The dataset list in the UI is empty, or I see "Insufficient permissions for datasets" in the logs. This actor needs Full permissions to see every dataset in your account — see the Security section above. Set it once in the Settings tab, then approve the one-time confirmation prompt on your next run.

I can't find the link to open the UI. It's printed once, right after the run starts — check the top of the Log tab for the actor run. It's also saved in the run's Output tab (the uiUrl field), so you can always go back and find it there if you scrolled past it in the logs.

---

Keywords: dedup datasets · deduplicate dataset · remove duplicate rows · merge datasets · merge csv files · clean data online · data cleaning tool · no-code data cleaning · csv cleaner · json cleaner · csv deduplication · upload csv · upload json · drag and drop csv · export excel · export csv · csv to excel · xlsx export · data wrangling · spreadsheet cleaner · flatten json · flatten nested json · apify actor · apify dataset tool · web scraping data cleanup · lukaskrivka dedup-datasets alternative

#DatasetCleaner #DataCleaning #Deduplication #CSV #JSON #Excel #NoCode #Apify #DataWrangling #WebScraping