# Merge Datasets & Remove Duplicates \[$0.20/1K 💰] (`cybermax/merge-dedupe-datasets`) Actor

💰 $0.20 per 1,000 unique rows; duplicates free, no start fee. Merge Apify datasets and remove duplicate rows by any fields (URL, name, phone, place ID) with smart matching that ignores case, spaces and URL variations. Cheaper than saliga's merge-dedupe ($0.50/1K). New rows only on schedules.

- **URL**: https://apify.com/cybermax/merge-dedupe-datasets.md
- **Developed by:** [CyberMax](https://apify.com/cybermax) (community)
- **Categories:** Developer tools, Automation, MCP servers
- **Features:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.20 / 1,000 unique row writtens

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Rowmeld: Merge & Deduplicate Datasets (Remove Duplicates)

#### Combine the results of many runs into one clean list, with every duplicate gone

Rowmeld merges any number of Apify datasets and removes duplicate rows by the fields you choose: URL, name, phone, place ID, or the whole row. **Smart matching** treats `https://www.acme.com/` and `acme.com`, `ACME Inc` and `acme inc`, or `+1 (555) 010-2000` and `555.010.2000` as the same. It can **fill blanks** from duplicates (one copy has the phone, another the email), **drop rows you already have**, and on schedules return **only rows it has never sent before**. **Duplicates are free; $0.20 per 1,000 unique rows.**

**[▶ Try it now](https://apify.com/cybermax/merge-dedupe-datasets)** (run it empty for a free demo on sample data) · $0.0002 per unique row (10,000 rows = $2) · no start fee · Apify's free plan includes $5 of credit a month

**Built for:** anyone who runs the same scraper many times (Google Maps leads by city, product or job searches by keyword) and needs one de-duplicated list · lead-list builders who must not pay twice for the same company · data teams joining exports before loading them into a CRM, Sheet or database · AI agents chaining Actors

**Real output** (demo run on Apify, 10 Nov 2026: two sample lists, 6 rows, deduplicated by `url` with fill blanks on, 1.8 s):

| name | url | city | phone |
|---|---|---|---|
| Blue Bottle Coffee | https://bluebottlecoffee.com/ | Oakland | +1 510-653-3394 *(filled from the duplicate http://www.bluebottlecoffee.com)* |
| Stumptown Coffee | https://www.stumptowncoffee.com | Portland | +1 503-230-7702 |
| Intelligentsia | https://intelligentsia.com | Chicago | +1 312-253-0594 *(filled from https://www.intelligentsia.com/)* |
| Counter Culture | https://counterculturecoffee.com/ | Durham | |

```json
{"datasetIds": ["DATASET_ID_1", "DATASET_ID_2"], "dedupeFields": ["url"]}
```

### How to use it

1. **Datasets to merge:** pick the datasets in the form (the results of your earlier runs), or pass their IDs through the API, e.g. a previous run's `defaultDatasetId`.
2. **Duplicate when these fields match:** for example `url`, or `name` + `address`, or `placeId`. Nested fields use dots: `location.city`. Leave empty to remove only rows that are identical in every field.
3. Run. The output dataset holds the unique rows; the run's `OUTPUT` record shows how many were read, removed, excluded and written.

### Options

| Option | What it does |
|---|---|
| `smartMatching` (on) | Ignores case and extra spaces; normalises URLs (scheme, `www.`, trailing slash) and phone numbers (digits only). Off = exact matching. |
| `keep` | Keep the `first` copy (earliest dataset and row) or the `last` (newest). |
| `fillBlanks` | Copy fields that are empty in the kept row from its duplicates. |
| `excludeDatasetIds` | Drop rows whose key appears in these datasets, e.g. leads you already contacted. |
| `outputFields` | Keep only these fields (dots for nested fields). |
| `addSourceDataset` | Adds `sourceDatasetId` to each row. |
| `keepItemsWithoutKey` | Rows whose dedupe fields are all empty can't be matched: keep (default) or drop them. |
| `maxItems` | Cap on rows written. |
| `monitor` + `monitorKey` | Remember every row written; later runs write only new rows (up to 1,000,000 remembered rows per watchlist). |

### Ready-made inputs

| Job | Input |
|---|---|
| **Merge Google Maps runs by place** | `{"datasetIds": ["…", "…"], "dedupeFields": ["placeId"], "fillBlanks": true}` |
| **One lead list by website** | `{"datasetIds": ["…", "…", "…"], "dedupeFields": ["website"]}` |
| **Only leads you don't have yet** | `{"datasetIds": ["new-run"], "excludeDatasetIds": ["my-crm-export"], "dedupeFields": ["website"]}` |
| **Newest price per product** | `{"datasetIds": ["monday", "tuesday"], "dedupeFields": ["url"], "keep": "last"}` |
| **Exact duplicates only** | `{"datasetIds": ["…"]}` |

### Run it on a schedule (only new rows)

Chain it after any scraper: in that scraper's **Integrations**, add "Run Actor" → `cybermax/merge-dedupe-datasets` with

```json
{"datasetIds": ["{{resource.defaultDatasetId}}"], "dedupeFields": ["url"], "monitor": true, "monitorKey": "my-leads"}
```

Each run then passes on only rows you have never received before, and you pay only for those.

### Integrations

- **API (curl):**
  ```bash
  curl -X POST "https://api.apify.com/v2/acts/cybermax~merge-dedupe-datasets/run-sync-get-dataset-items?token=YOUR_TOKEN" \
    -H "Content-Type: application/json" -d '{"datasetIds": ["ID1", "ID2"], "dedupeFields": ["url"]}'
  ```
- **Python:** `ApifyClient(token).actor("cybermax/merge-dedupe-datasets").call(run_input={"datasetIds": [run1["defaultDatasetId"], run2["defaultDatasetId"]], "dedupeFields": ["url"]})`
- **Make, Zapier, n8n:** run it after your scrapers, then send the unique rows to Google Sheets, Airtable, HubSpot or your database.
- **AI agents (MCP):** `https://mcp.apify.com?actors=cybermax/merge-dedupe-datasets`.

### Why Rowmeld

| | Rowmeld | Typical dedupe Actor on the Store |
|---|---|---|
| Access to your datasets | Uses Apify's dataset picker, so it works with today's limited-permission Actors | The most-used free one fails about 65% of its runs (Store stats, 10 Nov 2026) |
| Matching | Smart: case, spaces, URL and phone variations | Usually exact text |
| Fill blanks from duplicates | Yes | Rare |
| Exclude lists and new-rows-only runs | Yes | Rare |
| Price | **$0.20 per 1,000 unique rows; duplicates free** | $0.25–$2.50 per 1,000 rows on paid rivals (10 Nov 2026) |

### FAQ

**How big can the datasets be?** Millions of rows: rows stream through and only a short fingerprint of each key is kept (about 150 MB per million keys at the default 512 MB). "Fill blanks" keeps whole rows in memory, about 75,000 unique rows at 512 MB; give the run more memory for more.

**Can it read datasets from another Apify account?** Only datasets you can access. Pick them in the form or pass IDs from your own account.

**Does it change my original datasets?** No. It only reads them and writes a new dataset.

**Is my data safe?** Rowmeld reads the datasets you pick, writes the result to your own run, and sends nothing anywhere else.

### Also from CyberMax

- [Hookvolley: Bulk HTTP Requests & Webhook Sender](https://apify.com/cybermax/bulk-http-requests) · send the merged rows to any API, CRM or webhook.
- [Hollerdex: Website Contact Finder](https://apify.com/cybermax/website-contacts) · emails, phones and socials from company websites.
- [Namewhere: Company Name to Domain](https://apify.com/cybermax/company-to-domain) · find each company's website.

Found this useful? A quick rating helps others find it.

© 2026 CyberMax. All rights reserved.

# Actor input Schema

## `datasetIds` (type: `array`):

Pick the datasets (for example the results of several runs). Through the API, pass dataset IDs, e.g. a previous run's defaultDatasetId. Leave empty for a free demo on sample data.

## `dedupeFields` (type: `array`):

Fields that identify a row, e.g. url, or name + address, or placeId. Nested fields with dots: location.city. Empty = rows must be identical in every field.

## `smartMatching` (type: `boolean`):

Ignore case and extra spaces; treat https://www.site.com/ and site.com as the same; compare phone numbers by digits only.

## `keep` (type: `string`):

Which copy of a duplicate stays in the output.

## `fillBlanks` (type: `boolean`):

When the kept row has an empty field (no phone, no email) and a duplicate has it, copy it over. Holds rows in memory: about 75,000 unique rows at the default 512 MB; give the run more memory for more.

## `excludeDatasetIds` (type: `array`):

Optional. Rows whose key appears in these datasets are left out, e.g. leads you already have.

## `outputFields` (type: `array`):

Optional. Empty = all fields.

## `addSourceDataset` (type: `boolean`):

Adds sourceDatasetId to each row.

## `keepItemsWithoutKey` (type: `boolean`):

Rows where every dedupe field is empty can't be matched; keep them (on) or drop them (off).

## `maxItems` (type: `integer`):

Optional cap on rows written.

## `monitor` (type: `boolean`):

Remembers every row key this watchlist has written; later runs write only new rows. Rows sent before are not charged.

## `monitorKey` (type: `string`):

Optional name for this watchlist's saved state (use one per workflow).

## Actor input object example

```json
{
  "dedupeFields": [
    "url"
  ],
  "smartMatching": true,
  "keep": "first",
  "fillBlanks": false,
  "addSourceDataset": false,
  "keepItemsWithoutKey": true,
  "monitor": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "dedupeFields": [
        "url"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("cybermax/merge-dedupe-datasets").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "dedupeFields": ["url"] }

# Run the Actor and wait for it to finish
run = client.actor("cybermax/merge-dedupe-datasets").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "dedupeFields": [
    "url"
  ]
}' |
apify call cybermax/merge-dedupe-datasets --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cybermax/merge-dedupe-datasets"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/YHfCfLKciv0hCIivY/builds/nMDa6L4PFtXmFlVbn/openapi.json