You can access the Dataset Deduplicator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "buyerQuery": "buyer-ready records with source evidence",
10    "taskIntent": "buyer-ready-product-run",
11    "sourceMode": "sample",
12    "outputMode": "sample-records",
13    "startUrls": [],
14    "requestTimeoutSecs": 30,
15    "maxRequestRetries": 2,
16    "sinceLastRun": False,
17    "deltaMode": True,
18}
19
20# Run the Actor and wait for it to finish
21run = client.actor("zentrafoundry/dataset-deduplicator").call(run_input=run_input)
22
23# Fetch and print Actor results from the run's dataset (if there are any)
24print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
25for item in client.dataset(run.default_dataset_id).iterate_items():
26    print(item)
27
28# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Deduplicate Apify Datasets by URL, Email, Phone, or Fuzzy Ma API in Python

The Apify API client for Python is the official library that allows you to use Dataset Deduplicator API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Dataset Deduplicator API in JavaScript

Dataset Deduplicator API through CLI

Dataset Deduplicator OpenAPI definition

Dataset Deduplicator API

Deduplicate, Merge & Transform Datasets

datacach/deduplicate-datasets

Merge multiple datasets, deduplicate items by a combination of fields, and apply custom transforms — powered by Polars.

DataCach

Dataset Deduplicator & Cleaner

eszetael_lab/dataset-deduplicator-cleaner

Deduplicate a dataset: remove duplicates and merge duplicate records with exact, normalized or fuzzy matching. Clean scraped data — emails, phones, URLs, HTML — and get a data quality report. Runs on an Apify dataset ID or inline JSON, so it chains after any scraper.

Radosław Szal

Lead List Deduplicator & Merger

jurassic_jove/lead-deduplicator-merger

Merge and deduplicate lead lists from multiple Apify datasets, CSV files and inline JSON into one clean, outreach-ready list. Pure data processor — no scraping, no proxies, no external APIs.

Data Runner

Dataset Deduplicator

automation-lab/dataset-dedup

Merge and deduplicate Apify datasets by any field combination. Remove duplicates, keep first or last occurrence. Case-insensitive matching, whitespace trimming. Pay per 1K items processed.

Stas Persiianenko

Data Deduplicator

parsebird/dataset-deduplicator

Merge and deduplicate Apify datasets by any field combination. Remove duplicate rows while keeping the first or last occurrence. Supports case-insensitive matching and whitespace trimming.

ParseBird

JSON Dataset Cleaner and Deduplicator

rodrgds/dataset-cleaner

Clean JSON datasets, remove empty rows, deduplicate by any field, validate emails, and prepare scraper output for CRMs, analysis, or AI workflows.

Rodrigo Dias

Dataset Deduplicator v2

zentrafoundry/dataset-deduplicator-v2

Transform dataset deduplicator v2 inputs into structured rows, clear errors, confidence signals, and automation-ready output.

Zentra

Dataset Toolkit — Dedupe, Merge, Clean, Diff

glueworks/dataset-toolkit

Post-process any Apify dataset: remove duplicates, merge multiple datasets, clean and normalize fields, or diff two crawls. Outputs a new dataset plus a summary record.

KAHA Chen

Apify Dataset New Items Deduplicator

davidbenittah/apify-dataset-new-items-deduplicator

Return only never-before-seen rows from repeated Apify datasets and stop duplicate alerts, CRM records, and downstream automation actions.

David

Dataset Cleaner Visual Dedup Merge Export

leadsbrary/dataset-cleaner-visual-dedup-merge-export

Clean any Apify dataset visually: dedupe, filter, merge, rename/reorder columns, then export a new dataset, CSV, or XLSX. No code. Everything happens in a spreadsheet-style web UI.