# Dataset Transformer - Filter, Dedupe, Merge & Export CSV (`seemuapps/dataset-transformer`) Actor

Filter, deduplicate, merge, sort, rename and flatten Apify datasets from any scraper, then export clean rows to a dataset, CSV or JSON file.

- **URL**: https://apify.com/seemuapps/dataset-transformer.md
- **Developed by:** [Seemu Scraping](https://apify.com/seemuapps) (community)
- **Stats:** 3 total users, 2 monthly users, 66.7% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 row outputs

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Dataset Transformer - Filter, Dedupe, Merge & Export CSV

Clean up the results of any Apify scraper in one step: **filter rows, remove duplicates, merge several runs, keep or rename columns, flatten nested JSON, sort, and export a ready-to-use CSV**. No code, no spreadsheet formulas, and no downloading 50 MB of JSON to fix it by hand.

Point it at one or more datasets (or run IDs) and it writes the cleaned rows to a new dataset plus an `OUTPUT.csv` file you can open in Excel or Google Sheets.

### What you get

- **Filters** on any field, including nested fields (`address.city`) and lists (`posts.likes`): equals, not equals, contains, does not contain, regex, greater than / less than (numbers and dates), exists, does not exist, in list, not in list. Combine with AND or OR.
- **Smart number handling**: values scraped as text such as `"1,234"`, `"$19.99"`, `"12%"`, `"10.5K"` or `"2.1M"` compare as numbers.
- **Deduplication** by one or more fields (`phone`, `url`, `username`) or by the whole row. Loose matching ignores case and spacing and compares phone numbers by digits, so `(555) 123-4567` and `555.123.4567` count as the same lead.
- **Merge** any number of datasets in the order you list them, optionally tagging every row with the dataset it came from.
- **Column control**: keep only the fields you want (in your order), drop fields, rename fields, and flatten nested objects into `address.city`-style columns.
- **Sort** by any field, ascending or descending, then skip and limit rows.
- **Exports**: transformed rows in the run's dataset, `OUTPUT.csv` (always flat, RFC 4180 quoting, optional Excel BOM and `;` or tab delimiter) and optional `OUTPUT.json`.
- **Run summary** (`SUMMARY` record): rows read, rows filtered out, duplicates removed, rows output and download links.

### Use cases

- **Lead list cleanup**: dedupe Google Maps or directory scrapes by phone number or website before importing to a CRM.
- **Influencer and audience research**: keep only Instagram, TikTok or X profiles above a follower threshold, verified accounts, or bios matching a keyword.
- **Combine scheduled runs**: merge daily or weekly runs into one deduplicated master list.
- **Spreadsheet-ready exports**: flatten nested JSON and rename columns so the CSV drops straight into Google Sheets, Excel, Airtable or a mail-merge tool.
- **Competitor and price monitoring**: filter product or listing scrapes by price, rating, category or date and sort the result.

### How to use

1. Paste one or more **dataset IDs** (or run IDs, dataset names, or dataset URLs) into **Datasets to transform**. You can find them in the **Storage** tab of any finished run.
2. Optionally add **Filters**, **Remove duplicates by**, **Keep only these fields**, **Rename fields**, **Sort by** and **Max rows**.
3. Run the actor. Clean rows appear in the **Output** tab; `OUTPUT.csv` and `SUMMARY` are in the **Key-value store** tab.

#### Chain it after any scraper

To clean every run automatically, open your scraper, go to **Integrations**, add **Apify Actor**, choose Dataset Transformer, and set **Datasets to transform** to `{{resource.defaultDatasetId}}`. Every time the scraper finishes, the transformer runs on its results.

### Examples

#### Dedupe Google Maps leads by phone and export a CSV

```json
{
  "datasetIds": ["<your Google Maps run's dataset ID>"],
  "filters": [{ "field": "phone", "operator": "exists" }],
  "dedupeBy": ["phone"],
  "fields": ["title", "phone", "website", "address", "totalScore", "reviewsCount"],
  "renameFields": { "title": "businessName" },
  "sortBy": "reviewsCount",
  "sortOrder": "desc",
  "flatten": true
}
```

Sorting by `reviewsCount` descending before deduplication means that when the same phone appears twice, the listing with the most reviews is the one kept.

#### Instagram profiles with more than 10,000 followers

```json
{
  "datasetIds": ["<your Instagram profile scraper dataset ID>"],
  "filters": [
    { "field": "followersCount", "operator": "gt", "value": 10000 },
    { "field": "biography", "operator": "regex", "value": "coach|trainer|fitness" }
  ],
  "filterLogic": "AND",
  "dedupeBy": ["username"],
  "fields": ["username", "fullName", "followersCount", "biography", "externalUrl"]
}
```

`followersCount` values such as `"12.5K"` or `"1,200,000"` are compared as numbers.

#### Merge three runs into one CSV

```json
{
  "datasetIds": ["<run 1 dataset ID>", "<run 2 dataset ID>", "<run 3 dataset ID>"],
  "dedupeBy": ["url"],
  "addSourceField": true,
  "exportCsv": true,
  "csvBom": true
}
```

The output keeps the first occurrence of each URL in the order the datasets are listed, and `_sourceDataset` tells you which run each row came from.

#### More filter examples

| Goal | Rule |
|---|---|
| City is Austin (any case) | `{"field": "address.city", "operator": "equals", "value": "austin"}` |
| Has a website | `{"field": "website", "operator": "exists"}` |
| Email missing | `{"field": "email", "operator": "notExists"}` |
| Rating 4.5 or higher | `{"field": "totalScore", "operator": "gte", "value": 4.5}` |
| Posted in 2025 or later | `{"field": "createdAt", "operator": "gte", "value": "2025-01-01"}` |
| Country is US, CA or UK | `{"field": "country", "operator": "in", "value": ["US", "CA", "UK"]}` |
| Title does not mention "closed" | `{"field": "title", "operator": "notContains", "value": "closed"}` |
| Any post has over 1,000 likes | `{"field": "latestPosts.likes", "operator": "gt", "value": 1000}` |
| Exact-case match | `{"field": "sku", "operator": "equals", "value": "AB-12", "caseSensitive": true}` |

### Output format

The dataset contains your transformed rows, for example:

```json
{
  "businessName": "Blue Bottle Coffee",
  "phone": "+1 512-555-0134",
  "website": "https://bluebottlecoffee.com",
  "address.street": "123 Main St",
  "address.city": "Austin",
  "totalScore": 4.6,
  "reviewsCount": 1834
}
```

The `SUMMARY` record in the key-value store:

```json
{
  "sources": [{ "input": "aBcD1234", "datasetId": "aBcD1234", "rowsRead": 5000 }],
  "rowsRead": 5000,
  "rowsFiltered": 1210,
  "duplicatesRemoved": 344,
  "rowsSkippedByOffset": 0,
  "rowsOutput": 3446,
  "stoppedReason": null,
  "csvUrl": "https://api.apify.com/v2/key-value-stores/.../records/OUTPUT.csv",
  "jsonUrl": null
}
```

`stoppedReason` is `maxItems`, `maxChargeReached` or `timeout` when the run stopped early, and `null` otherwise.

### How the steps are applied

1. Datasets are read in the order listed (hidden `#fields` are skipped).
2. Filters run on the original rows, so you can filter on fields you later drop or rename.
3. If **Sort by** is set, rows are sorted; duplicates are then removed keeping the first row in sorted order.
4. **Skip first N rows** and **Max rows** are applied to the final result.
5. Fields are selected, removed, renamed and (optionally) flattened.

**Flattening and lists.** With the default *Join* mode, a list of plain values becomes one cell (`"Cafe, Coffee shop"`) and a list of objects becomes JSON text. *One column per item* gives `images.0.url`, `images.1.url` and so on. The CSV file is always flat, even when **Flatten nested objects** is off for the dataset.

### Limits and performance

- Without sorting, rows stream through in pages, so memory use stays flat for any dataset size. Deduplication keeps a short fingerprint per unique row (about 60 bytes), so 256 MB handles a few million unique keys.
- Sorting keeps a small index per row (about 250 bytes) and writes the rows to temporary disk, so 256 MB comfortably sorts several hundred thousand rows. For multi-million-row sorts, raise the run memory to 1 GB or more.
- Up to 50 datasets per run.
- If a run gets close to its timeout, it stops reading and still saves everything processed so far, including the CSV.

### FAQ

**Can I use a dataset from another account?** Yes, if that account has shared it with you or its storage access is public. Use the dataset ID or `username~dataset-name`.

**Does it change my original dataset?** No. The source datasets are only read; results go to this run's own dataset and key-value store.

**How do I open the CSV in Excel with accents and emoji intact?** Turn on **Excel-friendly CSV (UTF-8 BOM)**, and pick the semicolon delimiter if your Excel uses a decimal comma.

**What do I pay for?** One result event per row written to the output. Rows you filter out or remove as duplicates are not charged, and the run stops cleanly if it reaches your maximum cost per run.

# Actor input Schema

## `datasetIds` (type: `array`):

One or more dataset IDs, dataset names ('my-leads' or 'username~my-leads'), run IDs (the run's default dataset is used) or dataset/run URLs. Several datasets are merged in the order listed. When chaining from another actor via an integration, use {{resource.defaultDatasetId}}.

## `filters` (type: `array`):

Keep only rows that match these rules. Each rule: {"field": "dotted.path", "operator": "...", "value": ...}. Operators: equals, notEquals, contains, notContains, regex, notRegex, gt, gte, lt, lte, exists, notExists, in, notIn. Text matching is case-insensitive unless the rule sets "caseSensitive": true. Numbers stored as text ("1,234", "$19.99", "10.5K") and ISO dates compare correctly. If a path points into a list (e.g. "posts.likes"), the rule matches when any item matches. Leave empty to keep every row.

## `filterLogic` (type: `string`):

AND keeps rows that match every rule; OR keeps rows that match at least one rule.

## `dedupeBy` (type: `array`):

Field(s) that identify a duplicate, e.g. 'phone', 'url' or 'username'. With several fields, rows are duplicates only when all of them match. Use '\*' to remove rows that are identical in every field. The first occurrence is kept (after sorting, if a sort is set). Rows where all of these fields are empty are always kept. Leave empty to skip deduplication.

## `dedupeNormalize` (type: `boolean`):

Ignore case and extra spaces, treat '1,000' and 1000 as equal, and compare phone numbers by digits only ('(555) 123-4567' = '555.123.4567'). Turn off for exact matching.

## `fields` (type: `array`):

Output only these fields, in this order. Dotted paths reach into nested objects ('address.city'); a path through a list collects every item's value ('images.url'). Leave empty to keep all fields.

## `omitFields` (type: `array`):

Drop these fields from the output. Dotted paths are supported ('address.geo').

## `renameFields` (type: `object`):

JSON object of old name to new name, e.g. {"title": "businessName", "address.city": "city"}. Renaming a dotted path moves that value to the new top-level name.

## `flatten` (type: `boolean`):

Turn nested objects into dotted columns ({"address": {"city": "Austin"}} becomes "address.city": "Austin") in the dataset output too. The CSV file is always flat.

## `arrayMode` (type: `string`):

How lists become columns when flattening (and in the CSV file). Join: lists of text or numbers become 'a, b, c' and lists of objects become JSON text. Index: every item gets its own column (tags.0, tags.1, images.0.url). JSON: every list becomes JSON text.

## `sortBy` (type: `string`):

Sort the output by this field (dotted paths allowed). Numbers stored as text and ISO dates sort correctly; rows without the field go last. Leave empty to keep the input order.

## `sortOrder` (type: `string`):

Ascending (A to Z, smallest first) or descending (Z to A, largest first).

## `offset` (type: `integer`):

Skip this many rows of the final result (after filtering, deduplication and sorting).

## `maxItems` (type: `integer`):

Stop after this many output rows. 0 means no limit.

## `addSourceField` (type: `boolean`):

Add a '\_sourceDataset' field with the name or ID of the dataset each row came from. Useful when merging several runs.

## `pushToDataset` (type: `boolean`):

Write the transformed rows to this run's dataset, so you can export them in any format or chain another actor.

## `exportCsv` (type: `boolean`):

Also save all rows as OUTPUT.csv in the key-value store (always flat, with a header row).

## `exportJson` (type: `boolean`):

Also save all rows as a single OUTPUT.json array in the key-value store.

## `csvDelimiter` (type: `string`):

Column separator for OUTPUT.csv. Use semicolon for Excel in locales that use a decimal comma.

## `csvBom` (type: `boolean`):

Start OUTPUT.csv with a UTF-8 byte-order mark so Excel shows accents, emoji and non-Latin text correctly when the file is double-clicked.

## Actor input object example

```json
{
  "datasetIds": [
    "efewDi8Ra3E55LpJV"
  ],
  "filters": [
    {
      "field": "followersCount",
      "operator": "gt",
      "value": 10000
    },
    {
      "field": "isVerified",
      "operator": "equals",
      "value": true
    }
  ],
  "filterLogic": "AND",
  "dedupeBy": [],
  "dedupeNormalize": true,
  "fields": [],
  "omitFields": [],
  "renameFields": {
    "fullName": "name"
  },
  "flatten": false,
  "arrayMode": "join",
  "sortOrder": "asc",
  "offset": 0,
  "maxItems": 0,
  "addSourceField": false,
  "pushToDataset": true,
  "exportCsv": true,
  "exportJson": false,
  "csvDelimiter": ",",
  "csvBom": false
}
```

# Actor output Schema

## `results` (type: `string`):

One row per kept record, with your field selection, renames and flattening applied.

## `csv` (type: `string`):

OUTPUT.csv: every output row as a flat CSV with a header row.

## `json` (type: `string`):

OUTPUT.json: every output row as one JSON array (only when 'Save a JSON file' is on).

## `summary` (type: `string`):

SUMMARY: rowsRead, rowsFiltered, duplicatesRemoved, rowsOutput, per-source counts, CSV/JSON links and why the run stopped early, if it did.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "datasetIds": [
        "efewDi8Ra3E55LpJV"
    ],
    "filters": [
        {
            "field": "followersCount",
            "operator": "gt",
            "value": 10000
        },
        {
            "field": "isVerified",
            "operator": "equals",
            "value": true
        }
    ],
    "renameFields": {
        "fullName": "name"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("seemuapps/dataset-transformer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "datasetIds": ["efewDi8Ra3E55LpJV"],
    "filters": [
        {
            "field": "followersCount",
            "operator": "gt",
            "value": 10000,
        },
        {
            "field": "isVerified",
            "operator": "equals",
            "value": True,
        },
    ],
    "renameFields": { "fullName": "name" },
}

# Run the Actor and wait for it to finish
run = client.actor("seemuapps/dataset-transformer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "datasetIds": [
    "efewDi8Ra3E55LpJV"
  ],
  "filters": [
    {
      "field": "followersCount",
      "operator": "gt",
      "value": 10000
    },
    {
      "field": "isVerified",
      "operator": "equals",
      "value": true
    }
  ],
  "renameFields": {
    "fullName": "name"
  }
}' |
apify call seemuapps/dataset-transformer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,seemuapps/dataset-transformer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/I9V8AP5IaY1sm8xvX/builds/ZCTpp5JBpTQoebTtJ/openapi.json