# Dataset Cleaner & Exporter (`nerolabs/dataset-cleaner-exporter`) Actor

Deduplicate, flatten, clean and export any Apify dataset (or your own JSON) into a ready-to-use CSV and Excel file.

- **URL**: https://apify.com/nerolabs/dataset-cleaner-exporter.md
- **Developed by:** [Adam Pearce](https://apify.com/nerolabs) (community)
- **Categories:** Developer tools, Automation, Integrations
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 cleaned records

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### Turn any messy Apify dataset into a clean spreadsheet

Just finished a scrape and got back a pile of nested JSON with duplicate rows, inconsistent casing, and half-empty fields? **Dataset Cleaner & Exporter** takes any Apify dataset (or your own JSON) and hands back a deduplicated, flattened, cleaned dataset plus a ready-to-open **CSV and Excel file**, no manual export-and-tidy-up step required.

Point it at a `sourceDatasetId` from any other Actor's run, or paste inline JSON, and it will:

- **Deduplicate** exact, normalized, or fuzzy near-duplicates, keeping the most complete record
- **Flatten** nested objects into spreadsheet-ready columns (`address.city` becomes `address_city`)
- **Clean** emails, phone numbers and URLs, strip HTML, trim whitespace, coerce types
- **Explode** one named array field (e.g. `offers` or `variants`) into multiple rows, if you want that
- **Export** a real downloadable CSV and XLSX file, not just a dataset you have to know how to export yourself

### Why use this instead of exporting manually?

Apify's own dataset export already gives you a raw CSV or JSON, but it's still the dataset you started with: duplicates and all, nested fields collapsed into unreadable JSON strings, no cleanup. This Actor is the step most people do by hand in Excel or a spreadsheet formula afterwards, done automatically, and chainable straight after any scraper in a pipeline or schedule.

- **Data teams**: standardize scraper output before loading into a database or BI tool
- **Lead generation**: dedupe and normalize contact lists from multiple scraper runs
- **Automation builders**: drop this in as a cleanup step between a scraper and a Google Sheets/Slack/webhook integration
- **AI agents**: a deterministic, no-hallucination-risk primitive for "clean this data" as part of a larger pipeline

### How to use it

1. Run any scraper or data-producing Actor and copy its dataset ID from the Storage tab (or grab `defaultDatasetId` from its API response).
2. Run this Actor with that dataset ID in **Source dataset**, or paste your own JSON array into **Inline data**.
3. Pick the cleaning options you want (deduplication mode, flatten, field cleaning) or leave the sensible defaults.
4. Download the CSV or Excel file from the run's **Output** tab, or read the cleaned dataset via the API for further automation.

### Input

The full list of options is in the **Input** tab. The two you'll use most:

- **Source dataset**: pick an existing Apify dataset from any of your runs.
- **Inline data**: paste a JSON array instead, for ad-hoc or piped data.

Everything else (deduplication mode, flatten, field cleaning, column keep/remove/rename, export formats) has a sensible default, so you can run it with just a dataset picked and get a good result.

### Output

Three things land in every run:

1. **Cleaned dataset** (the Actor's own dataset): the same records, deduplicated, flattened and cleaned.
2. **CSV and/or Excel file**, in the run's key-value store, linked directly from the **Output** tab.
3. **Quality report** (`QUALITY_REPORT` in the key-value store): input/output counts, duplicates removed, per-field fill rates, and any warnings from the run.

Example cleaned record:

```json
{
  "Company Name": "Acme Corp",
  "Email": "sales@acme.com",
  "Phone": "+15551234567",
  "Website": "https://acme.com",
  "Details_hq_city": "Austin",
  "Details_hq_state": "TX"
}
```

### Pricing

Pay per event, no subscription:

- **$0.002 per cleaned record** written to the output dataset
- **$0.01 per exported file** (CSV and Excel each count once, so both together is $0.02 per run)

A realistic run cleaning 2,000 scraped leads with both CSV and Excel export costs roughly **$4.02**. A quick one-off cleanup of 50 records with just a CSV export costs about **$0.11**. There's no charge for records removed as duplicates, only for what actually ends up in your cleaned output.

### Tips

- If you're not sure what a source dataset looks like, run it with **Max output records** set low (e.g. 20) first to check the result before running the full dataset.
- Combining **Field to expand into rows** with deduplication needs a small adjustment: if you explode an array field (like `offers`) into multiple rows and then deduplicate, make sure your **Deduplication keys** include a field that varies per exploded row (like `plan` or `sku`), not just a field shared across every row of that record (like `email`). Otherwise rows that only differ in the exploded field will be treated as duplicates of each other.
- Fuzzy deduplication is recommended for datasets under 5,000 records. Above that, it automatically falls back to normalized matching so the run still completes reliably, and the quality report tells you when that happened.

### FAQ

**Does this work with the output of any Actor, or just specific ones?** Any Actor's dataset. It makes no assumptions about field names or shape, it works on whatever JSON structure your source dataset actually has.

**Will it invent or guess data?** No. Every operation is deterministic (no AI/LLM in the loop), and anything it can't confidently interpret (an unparseable phone number, an array it wasn't told to expand) is left as-is or reported in the quality report rather than guessed at.

**What happens to duplicate records, are they billed?** No, you're only billed for records that end up in the cleaned output. Records removed as duplicates cost nothing.

**Can I run this on a schedule after another Actor?** Yes, that's the intended pattern: schedule your scraper, then schedule this Actor a few minutes later pointed at the same dataset, or chain them via an integration.

If this saved you a manual export-and-clean-up pass, a review on this page helps a lot. Found a bug or have a request? Use the Issues tab, replies are personal, not automated.

# Actor input Schema

## `sourceDatasetId` (type: `string`):

Pick an existing Apify dataset to clean, for example the output of another Actor run. Use this OR 'Inline data' below, not both. Declaring it this way is what lets this Actor run with limited permissions: it may read the dataset you point at, and nothing else on your account.

## `data` (type: `array`):

A JSON array of records to clean, for ad-hoc or piped data instead of a dataset ID.

## `flatten` (type: `boolean`):

Turn nested objects into flat scalar columns (e.g. address.city becomes address\_city) so the result is spreadsheet-ready. Arrays are kept as a single JSON-text cell unless you set 'Field to expand into rows' below.

## `flattenSeparator` (type: `string`):

Character(s) used to join nested key paths when flattening.

## `expandArrayField` (type: `string`):

Optional. Name one top-level array-of-objects field (e.g. 'offers' or 'variants') to explode into multiple output rows, one per entry, with the rest of the record repeated. Leave empty to keep arrays as a single cell.

## `dedupMode` (type: `string`):

How to detect duplicate records. Exact = byte-identical values. Normalized = ignores case/whitespace differences. Fuzzy = also merges near-duplicates by similarity (recommended for datasets under 5,000 records). None = skip deduplication.

## `dedupKeys` (type: `array`):

Field names that identify a duplicate (e.g. email, url). Leave empty to compare the whole record.

## `similarityThreshold` (type: `number`):

Fuzzy mode only. 0.5 to 0.99. Higher means stricter, fewer records get merged. 0.9 is a good starting point.

## `keepStrategy` (type: `string`):

When duplicates are found, which record survives.

## `cleanFields` (type: `boolean`):

Auto-detect email, phone and URL fields by name or shape, and normalize them (lowercase emails, digit-only phones, lowercase URL host).

## `stripHtml` (type: `boolean`):

Remove HTML tags and decode common entities from text fields.

## `coerceTypes` (type: `boolean`):

Convert numeric-looking and true/false-looking strings into real numbers and booleans.

## `emptyToNull` (type: `boolean`):

Turn empty or blank string values into null for consistency.

## `dropEmptyFields` (type: `boolean`):

Remove null/empty fields from each record entirely instead of keeping them.

## `columnsToKeep` (type: `array`):

Only these columns will be kept in the output (takes priority over 'Columns to remove'). Leave empty to keep all.

## `columnsToRemove` (type: `array`):

These columns will be dropped from the output. Ignored if 'Columns to keep' is set.

## `columnRenameMap` (type: `array`):

Rename columns, one per line, in 'oldName:newName' format (e.g. 'address\_city:city').

## `exportFormats` (type: `array`):

Which downloadable file(s) to generate alongside the cleaned dataset.

## `maxItems` (type: `integer`):

Cap on output records (0 = no cap, up to the safety limit of 100,000). You are billed per output record and per exported file, so capping this caps your cost.

## Actor input object example

```json
{
  "data": [
    {
      "Company Name": "  Acme Corp  ",
      "Email": "SALES@ACME.COM",
      "Phone": "+1 (555) 123-4567",
      "Website": "https://Acme.com/"
    },
    {
      "Company Name": "Acme Corp",
      "Email": "sales@acme.com",
      "Phone": "555.123.4567",
      "Website": "https://acme.com"
    },
    {
      "Company Name": "Globex Inc",
      "Email": "info@globex.io",
      "Phone": "+1 555 987 6543",
      "Website": "https://globex.io/",
      "Details": {
        "founded": 1998,
        "tags": [
          "b2b",
          "logistics"
        ]
      }
    }
  ],
  "flatten": true,
  "flattenSeparator": "_",
  "dedupMode": "normalized",
  "dedupKeys": [
    "Email"
  ],
  "similarityThreshold": 0.9,
  "keepStrategy": "most_complete",
  "cleanFields": true,
  "stripHtml": false,
  "coerceTypes": false,
  "emptyToNull": true,
  "dropEmptyFields": false,
  "exportFormats": [
    "csv",
    "xlsx"
  ],
  "maxItems": 0
}
```

# Actor output Schema

## `cleanedRecords` (type: `string`):

The deduplicated, flattened and cleaned records.

## `csvFile` (type: `string`):

A ready-to-open CSV file of the cleaned data.

## `xlsxFile` (type: `string`):

A ready-to-open Excel (.xlsx) file of the cleaned data.

## `qualityReport` (type: `string`):

Input/output counts, duplicates removed, per-field fill rates and any warnings from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "data": [
        {
            "Company Name": "  Acme Corp  ",
            "Email": "SALES@ACME.COM",
            "Phone": "+1 (555) 123-4567",
            "Website": "https://Acme.com/"
        },
        {
            "Company Name": "Acme Corp",
            "Email": "sales@acme.com",
            "Phone": "555.123.4567",
            "Website": "https://acme.com"
        },
        {
            "Company Name": "Globex Inc",
            "Email": "info@globex.io",
            "Phone": "+1 555 987 6543",
            "Website": "https://globex.io/",
            "Details": {
                "founded": 1998,
                "tags": [
                    "b2b",
                    "logistics"
                ]
            }
        }
    ],
    "dedupKeys": [
        "Email"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("nerolabs/dataset-cleaner-exporter").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "data": [
        {
            "Company Name": "  Acme Corp  ",
            "Email": "SALES@ACME.COM",
            "Phone": "+1 (555) 123-4567",
            "Website": "https://Acme.com/",
        },
        {
            "Company Name": "Acme Corp",
            "Email": "sales@acme.com",
            "Phone": "555.123.4567",
            "Website": "https://acme.com",
        },
        {
            "Company Name": "Globex Inc",
            "Email": "info@globex.io",
            "Phone": "+1 555 987 6543",
            "Website": "https://globex.io/",
            "Details": {
                "founded": 1998,
                "tags": [
                    "b2b",
                    "logistics",
                ],
            },
        },
    ],
    "dedupKeys": ["Email"],
}

# Run the Actor and wait for it to finish
run = client.actor("nerolabs/dataset-cleaner-exporter").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "data": [
    {
      "Company Name": "  Acme Corp  ",
      "Email": "SALES@ACME.COM",
      "Phone": "+1 (555) 123-4567",
      "Website": "https://Acme.com/"
    },
    {
      "Company Name": "Acme Corp",
      "Email": "sales@acme.com",
      "Phone": "555.123.4567",
      "Website": "https://acme.com"
    },
    {
      "Company Name": "Globex Inc",
      "Email": "info@globex.io",
      "Phone": "+1 555 987 6543",
      "Website": "https://globex.io/",
      "Details": {
        "founded": 1998,
        "tags": [
          "b2b",
          "logistics"
        ]
      }
    }
  ],
  "dedupKeys": [
    "Email"
  ]
}' |
apify call nerolabs/dataset-cleaner-exporter --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nerolabs/dataset-cleaner-exporter"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/F5fhHE0Idn0w3BbNm/builds/mqVCKKB7VwCsROHkt/openapi.json