Pricing

Pay per usage

Try for free

Go to Apify Store

Dataset Deduplicator & Cleaner

Try for free

Remove duplicate rows from an Apify Dataset or inline JSON. Deduplicate by selected fields, trim strings, and write the cleaned rows to a new Dataset.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Starshape Tools

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

What it does

Reads rows from an Apify Dataset or inline JSON
Deduplicates by one or more selected fields
Can keep the first or last duplicate
Optionally trims leading/trailing whitespace from text values
Optionally compares text keys case-insensitively
Writes the cleaned rows to the default Apify Dataset

Quick test

Run the Actor with the default input.

The sample contains 3 rows, including two rows with id = A001. With the default keyFields = ["id"] and keep = first, the output contains 2 rows.

Example

Input:

[
  {"id":"A001","name":" Alpha ","score":10},
  {"id":"A002","name":"Beta","score":20},
  {"id":"A001","name":"Alpha duplicate","score":99}
]

Output:

[
  {"id":"A001","name":"Alpha","score":10},
  {"id":"A002","name":"Beta","score":20}
]

Dataset mode

Choose a Dataset in Source Dataset. The Actor reads up to maxItems rows from it and writes the cleaned result to the run's default Dataset.

Deduplication keys

Set Deduplication key fields to fields such as:

email

or:

firstName
lastName
company

Leave the list empty to compare the complete cleaned row.

Notes

The source Dataset is read-only.
The Actor runs with limited permissions and requests READ access only to a Dataset explicitly selected by the user.
maxItems is a safety limit, not pagination across multiple runs.
String trimming applies to top-level fields only.

Dataset Deduplicator & Cleaner

eszetael_lab/dataset-deduplicator-cleaner

Returns your records with duplicates removed and values normalized — same fields as the input, nothing added or renamed. Exact, normalized or fuzzy matching on keys you choose. A data quality report lands under QUALITY_REPORT. Takes a dataset ID or inline JSON, so it chains after any scraper.

Radosław Szal

Dataset Cleaner & Deduplicator

luminous_i/dataset-cleaner-deduplicator

Clean, normalize and deduplicate any dataset: field names, whitespace, empty values, numbers, plus exact and fuzzy duplicate removal.

Steve

Dataset Cleaner & Deduplicator — Quality Engine

egeusta/dataset-quality-deduplication-engine

Clean datasets, remove duplicate rows, detect missing values, normalize emails and phones, and validate schemas before exporting to CRM or databases.

Ege Usta

Playwright Scraper — Python, Any Website

eszetael_lab/reliable-playwright-scraper

Scrape any JavaScript-heavy site with Playwright and a Python page function. Pages that returned nothing are marked in the data and cost you nothing — you pay per delivered record, never per run. Every record says whether the page loaded, timed out or was blocked.

Radosław Szal

JSON Dataset Cleaner and Deduplicator

rodrgds/dataset-cleaner

Clean JSON datasets, remove empty rows, deduplicate by any field, validate emails, and prepare scraper output for CRMs, analysis, or AI workflows.

Rodrigo Dias

Review Cleaner & Deduplicator

om_kh/reviews-cleaner-deduplicator

Normalizes review rows from any source into one clean schema: rating, title, text, author, date, id. Duplicates removed. Diff two pulls to isolate new reviews. First 10 results free.

omar khandji

Data Deduplicator

parsebird/dataset-deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

ParseBird

Data Cleaner

parsebird/data-cleaner

Clean messy data — remove nulls, normalize case, trim whitespace, format phone numbers and emails, extract domains, convert types, and more. Works with Apify datasets or direct JSON input.

ParseBird

HTTP Request

parsebird/http-request-actor

Send HTTP requests (GET, POST, PUT, PATCH, DELETE) to any API endpoint with customizable headers, parameters, and body data. Forward responses via callback URL. Perfect for Clay users and no-code API integrations.

ParseBird

180

5.0

(1)

Lead List Cleaner - Dedupe, Normalize & Merge for CRM

webdata_labs/lead-list-deduplicator

[💵 $0.05 / 1K] Clean messy B2B lead lists into CRM-ready company/contact records with duplicate clusters, confidence scores, match reasons, normalized domains, emails, and phones.

WebData Labs

Lead Cleaner - Fuzzy Dedupe & Data Normalizer

javitax47/lead-cleaner

This Actor merges leads like "My Company, S.L." and "MY COMPANY SL" and explains why. It normalizes phones (E.164), strips 40+ EU legal forms, folds Gmail aliases, and validates NIF/CIF/VAT check digits before deduplicating. Includes a labelled 63-row benchmark you can run yourself.