# CSV Merge & Deduplicate (`danielmichalek/csv-merge-deduplicate`) Actor

Merge CSV exports, align columns by name and remove duplicate rows. Get a clean CSV plus a mismatch report. Auto-detects separators, supports aliases and escapes spreadsheet formulas. Up to 20 files per merge. No AI model or external website access.

- **URL**: https://apify.com/danielmichalek/csv-merge-deduplicate.md
- **Developed by:** [Daniel Michałek](https://apify.com/danielmichalek) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$20.00 / 1,000 completed csv merges

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## CSV Merge & Deduplicate

Combine differently shaped CSV exports into one consistent CSV without manually rearranging columns. Useful for recurring exports from shops, CRMs, reporting tools and spreadsheets.

### What this Actor does

- Aligns columns by name, ignoring case, accents and punctuation.
- Supports explicit aliases, such as `Amount` for `Kwota` or `Betrag`.
- Detects comma, semicolon, tab and pipe delimiters separately for each file.
- Optionally removes identical aligned rows, ignoring the generated source filename.
- Adds a source filename column and reports missing columns, renamed headers and inconsistent rows.
- Preserves quoted delimiters and multiline fields. Rejects unclosed quotes rather than silently returning damaged output.
- Escapes potential spreadsheet formulas by default. Supports a UTF-8 BOM for Excel.

No language model, proxy, target website login or additional API key is required. Processing runs on Apify. Inputs and outputs are stored by Apify under your account's storage and retention settings; this is not an offline or zero-retention service.

### Example input

```json
{
  "files": [
    {"name": "january.csv", "content": "Name,Amount\nAlice,10\n"},
    {"name": "february.csv", "content": "amount;name;City\n20;Bob;Berlin\n"}
  ],
  "dropDuplicateRows": true,
  "outputBom": true
}
```

Result, without the optional BOM shown:

```csv
Name,Amount,City,source_file
Alice,10,,january.csv
Bob,20,Berlin,february.csv
```

For explicit aliases, add `"aliases": {"Amount": ["Kwota", "Betrag"]}`. Set `"sourceColumn": ""` to omit the generated filename column.

### Output and download

The default dataset contains **one object for the whole completed merge**, not one object per CSV row. This object contains `mergedCsv`, `rowCount`, `columnCount`, `header`, `inputHash` and `report`.

The default key-value store contains:

- `OUTPUT.csv`: the merged UTF-8 CSV, ready to download.
- `REPORT`: a JSON report with per-file details and warnings.

To download the actual merged file, use `OUTPUT.csv` in the run's key-value store. Exporting the dataset as CSV instead produces a one-record summary that embeds the merged text. If a download mirror cannot be saved, the complete `mergedCsv` remains available in the dataset.

### Pricing model

The Actor is designed for the **merge-completed** pay-per-event price shown in the Pricing tab. One successful, non-empty merged result is one event, whether it combines one file or twenty. There is no per-input-row fee and the Actor does not emit a paid startup event. A header-only input emits no completed-merge event. A maximum-charge limit that accepts no event prevents delivery of an unpaid result. Actual billing depends on the active platform pricing configuration; verify the current Pricing tab before running.

After a platform restart, an existing matching result in the same dataset is reused without emitting another result event. Starting a separate new run is a new merge. Following an interrupted or failed paid storage operation, inspect the existing run before retrying; no claim of transactional exactly-once billing across every failure mode is made.

### Limits and scope

Maximums per run: 20 files; 1 MiB of combined UTF-8 CSV text; 10,000 input data rows; 100 input columns in the combined schema; 100,000 projected aligned output cells, including the optional source column; 4 MiB output CSV. Additional structural and metadata limits protect memory. Oversized or malformed inputs are rejected, not silently truncated. Limits apply before deduplication.

This version accepts CSV **text in JSON**, not file paths, remote URLs, XLSX workbooks or archives. File names are labels only and must be unique, without directory paths. Supply Unicode text; legacy binary encodings must be decoded before calling this Actor.

Header matching is intentionally aggressive: for example, `Invoice No` and `invoice_no` are treated as the same column. Review the report if your source systems use similar names for different concepts. Deduplication compares whole rows; it is not a database join or a customer-record identity matcher. Empty and repeated header rows are skipped and documented in the report. Additional fields can become generated columns, while missing fields stay empty.

Formula escaping prepends an apostrophe to risky output values, including negative numeric-looking values. This favors safe spreadsheet opening over automatic numeric interpretation. Turn it off only for trusted data when your downstream importer requires numbers. A CSV file itself contains text rather than a guaranteed spreadsheet cell type.

### Support

Use the Issues tab for a reproducible example containing synthetic or anonymized data. Do not post confidential exports, passwords, tokens or personal customer records in a public issue.

# Actor input Schema

## `files` (type: `array`):

An array of objects with a unique name and CSV text in content. CSV line breaks must be escaped as \n in JSON. Total CSV text is limited to 1 MiB.

## `aliases` (type: `object`):

Map canonical names to variants, e.g. {"Amount":\["Kwota","Betrag"]}. Matching already ignores letter case, accents and punctuation. Ambiguous mappings are rejected.

## `inputDelimiter` (type: `string`):

Auto detects the separator for each file separately. An Excel sep= directive is supported.

## `outputDelimiter` (type: `string`):

Choose the separator for the merged CSV.

## `sourceColumn` (type: `string`):

Adds the input filename to every result row. Set to an empty string to disable this column.

## `dropDuplicateRows` (type: `boolean`):

Deduplicate complete aligned rows, ignoring the generated source filename. First occurrence wins. This does not merge records by a business key.

## `trimValues` (type: `boolean`):

Remove leading and trailing whitespace in values before matching duplicate rows.

## `outputBom` (type: `boolean`):

Adds a UTF-8 byte-order mark to help Excel recognize accented characters.

## `escapeFormulas` (type: `boolean`):

Prepend an apostrophe to potentially active spreadsheet formulas in output cells and headers. This also turns negative numeric-looking values into spreadsheet text. Disable only for trusted data when you need numeric interpretation.

## Actor input object example

```json
{
  "files": [
    {
      "name": "january.csv",
      "content": "Name,Amount\nAlice,10\n"
    },
    {
      "name": "february.csv",
      "content": "amount;name;City\n20;Bob;Berlin\n"
    }
  ],
  "aliases": {},
  "inputDelimiter": "auto",
  "outputDelimiter": ",",
  "sourceColumn": "source_file",
  "dropDuplicateRows": false,
  "trimValues": false,
  "outputBom": false,
  "escapeFormulas": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `mergedCsv` (type: `string`):

No description

## `report` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "files": [
        {
            "name": "january.csv",
            "content": "Name,Amount\nAlice,10\n"
        },
        {
            "name": "february.csv",
            "content": "amount;name;City\n20;Bob;Berlin\n"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("danielmichalek/csv-merge-deduplicate").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "files": [
        {
            "name": "january.csv",
            "content": """Name,Amount
Alice,10
""",
        },
        {
            "name": "february.csv",
            "content": """amount;name;City
20;Bob;Berlin
""",
        },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("danielmichalek/csv-merge-deduplicate").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "files": [
    {
      "name": "january.csv",
      "content": "Name,Amount\\nAlice,10\\n"
    },
    {
      "name": "february.csv",
      "content": "amount;name;City\\n20;Bob;Berlin\\n"
    }
  ]
}' |
apify call danielmichalek/csv-merge-deduplicate --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,danielmichalek/csv-merge-deduplicate"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hDb2mSsoKv2VbAwwC/builds/AIIOoruSJjdgggwkM/openapi.json