Under maintenance

Pricing

Pay per event

Try for free

Go to Apify Store

Dataset Deduplicator

Under maintenance

Try for free

Merge and deduplicate Apify datasets by any field combination. Remove duplicates, keep first or last occurrence. Case-insensitive matching, whitespace trimming. Pay per 1K items processed.

Pricing

Pay per event

Rating

0.0

(0)

Developer

Stas Persiianenko

Actor stats

Bookmarked

Total users

Monthly active users

an hour ago

Last modified

Categories

Developer tools

You can access the Dataset Deduplicator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "datasetIds": [
12        "a8HvXCQBgz2jolQa4"
13    ],
14    "fields": [
15        "url"
16    ],
17    "maxItems": 10000
18};
19
20// Run the Actor and wait for it to finish
21const run = await client.actor("automation-lab/dataset-dedup").call(input);
22
23// Fetch and print Actor results from the run's dataset (if any)
24console.log('Results from dataset');
25console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
26const { items } = await client.dataset(run.defaultDatasetId).listItems();
27items.forEach((item) => {
28    console.dir(item);
29});
30
31// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Dataset Deduplicator — Remove Duplicate Data Records API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Dataset Deduplicator API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Dataset Deduplicator API in Python

Dataset Deduplicator API through CLI

Dataset Deduplicator OpenAPI definition

Dataset Deduplicator API

Contact Details Merge & Deduplicate

lukaskrivka/contact-details-merge-deduplicate

Merge and deduplicate all contacts extracted by Contact Details Scraper. Works with multiple datasets. One row per domain.

Lukáš Křivka

119

Filter dataset records

analogous_ottoman/filter-records-based-on-negative-keywords

This actor lets you select a field in your dataset and exclude some records if they contain a keyword in the list of excluded keywords you provide (case-insensitive).

Analogous

Fast Dataset Cleaner & CSV Formatter

motivational_nickel/dataset-cleaner-and-formatter

Fast dataset cleaning for CSV and JSON files. Automatically removes duplicates, trims whitespace, fixes capitalization, and normalizes fields. Works with Apify datasets or uploaded files and prepares data for analytics, CRM imports, and automation pipelines.

Leoncio Jr Coronado

🔥 Power Data Transformer

wiseek/power-data-transformer

🔥 Unlock your scraped data—clean, merge, split, deduplicate, filter, standardize, validate, enrich and sync—using built-in transformations and powerful SQL pipelines for ETL/ELT workflows. Seamlessly integrate processed datasets with automation platforms like n8n, Make.com, and Zapier.

wiseek

Sort Dataset Items

lukaskrivka/sort-dataset-items

Add this actor as a webhook to your scraper to sort the dataset by index field

Lukáš Křivka

Google Dataset Items Translator

web.harvester/google-dataset-items-translator

Translate any dataset field(s) to any of the supported languages using the Google Translate website, it goes through all the items in the dataset and translates all of the selected fields

Web Harvester

Text Case Converter

automation-lab/text-case-converter

This actor converts text between multiple case formats: UPPER CASE, lower case, Title Case, camelCase, PascalCase, snake_case, kebab-case, and CONSTANT_CASE. Automatically splits words from any input format. Useful for data transformation, code generation, and bulk text processing.

Stas Persiianenko

Scraped Data Cleaner & Converter (No-Code CSV/JSON Tool) Rental

m3web/scraped-data-cleaner-rental

Clean and organize scraped .json or .csv data — no coding required. Remove duplicates, empty rows, unwanted columns, and sort by any field. Cleaned results are pushed to your Apify dataset. Perfect for marketers, researchers, and no-code workflows.