Pricing

from $1.49 / 1,000 items processeds

Data Deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

Pricing

from $1.49 / 1,000 items processeds

Rating

0.0

(0)

Developer

ParseBird

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

Categories

Automation

Developer tools

You can access the Data Deduplicator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "jsonData": [
10        {
11            "name": "John Doe",
12            "email": "john@example.com",
13            "company": "Acme",
14        },
15        {
16            "name": "Jane Smith",
17            "email": "jane@example.com",
18            "company": "Beta",
19        },
20        {
21            "name": "John D.",
22            "email": "john@example.com",
23            "company": "Acme Inc",
24        },
25    ],
26    "fields": ["email"],
27}
28
29# Run the Actor and wait for it to finish
30run = client.actor("parsebird/dataset-deduplicator").call(run_input=run_input)
31
32# Fetch and print Actor results from the run's dataset (if there are any)
33print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
34for item in client.dataset(run["defaultDatasetId"]).iterate_items():
35    print(item)
36
37# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Data Deduplicator — Merge & Deduplicate Apify Datasets API in Python

The Apify API client for Python is the official library that allows you to use Data Deduplicator API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Data Deduplicator API in JavaScript

Data Deduplicator API through CLI

Data Deduplicator OpenAPI definition

Data Deduplicator API

Dataset Deduplicator

automation-lab/dataset-dedup

Merge and deduplicate Apify datasets by any field combination. Remove duplicates, keep first or last occurrence. Case-insensitive matching, whitespace trimming. Pay per 1K items processed.

Stas Persiianenko

CSV Diff Tool

automation-lab/csv-diff-tool

Compare two CSV datasets and find added, removed, and modified rows. Supports key-column matching, configurable delimiters, case-insensitive comparison, and whitespace trimming. Exports a structured change report with before/after values.

Stas Persiianenko

Data Cleaner

parsebird/data-cleaner

Clean messy data — remove nulls, normalize case, trim whitespace, format phone numbers and emails, extract domains, convert types, and more. Works with Apify datasets or direct JSON input.

ParseBird

🧼 Scraped Data CSV Cleaner

taroyamada/csv-data-cleaner

Polish raw outputs from Google Maps and Instagram profile scrapers. Merge duplicate contacts, clear empty spreadsheet rows, and sort email lists automatically.

太郎山田

Filter dataset records

analogous_ottoman/filter-records-based-on-negative-keywords

This actor lets you select a field in your dataset and exclude some records if they contain a keyword in the list of excluded keywords you provide (case-insensitive).

Analogous

Text Case Converter

automation-lab/text-case-converter

This actor converts text between multiple case formats: UPPER CASE, lower case, Title Case, camelCase, PascalCase, snake_case, kebab-case, and CONSTANT_CASE. Automatically splits words from any input format. Useful for data transformation, code generation, and bulk text processing.

Stas Persiianenko

Glassdoor Jobs | Remove Duplicate Jobs | Cheapest

cheap_scraper/glassdoor-jobs-scraper-remove-duplicate-jobs

Glassdoor Jobs | Remove Duplicate Jobs | Cheapest The Glassdoor jobs scraper allows you to collect jobs By entering multiple keywords, search queries.

cheap_scraper

356

LinkedIn Jobs Scraper | Remove Duplicate Jobs | Pay Per Result

cheap_scraper/linkedin-job-scraper

LinkedIn Jobs Scraper | Remove Duplicate Jobs. The LinkedIn jobs scraper allows you to collect jobs in 2 ways: By providing one or more start URLs, or By entering multiple keywords, search queries. You can use either method individually or combine both.

cheap_scraper

2.9K

4.7

Scraped Data Cleaner & Converter (No-Code CSV/JSON Tool) Rental

m3web/scraped-data-cleaner-rental

Clean and organize scraped .json or .csv data — no coding required. Remove duplicates, empty rows, unwanted columns, and sort by any field. Cleaned results are pushed to your Apify dataset. Perfect for marketers, researchers, and no-code workflows.