# CSV Cleaner — typed sniff, normalise, dedupe (`draeg82/csv-cleaner`) Actor

Deterministic CSV cleaning for agent pipelines: typed column sniffing, header normalisation, validation with per-row drop reasons, and exact/near dedupe. Zero network at runtime, zero keys, pure local computation. Every drop is reported; values are never guessed or fuzzy-rewritten.

- **URL**: https://apify.com/draeg82/csv-cleaner.md
- **Developed by:** [Andy Mitchell](https://apify.com/draeg82) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## csv-cleaner

Deterministic typed sniff / normalise / dedupe for messy CSV — built for agent pipelines
that receive dirty lead lists and need machine-readable cleaning reports, not guesses.

Pure local computation: zero network at runtime, zero keys, zero scraping.

### What it does

- **Parse** CSV text, an array of line strings, or a `{"rows": [...]}` object array (serialised
  to CSV text with a header from the union of keys before parsing — same clean output as the
  equivalent CSV string): quoted fields, CRLF, BOM stripping,
  blank-line skipping, encoding normalisation to trimmed UTF-8 strings.
- **Header normalisation**: trim, lowercase, collapse internal whitespace, dedupe collisions.
- **Typed column sniffing** per column: `int | float | date | postcode | email | url | bool | text`,
  with a confidence score (share of unambiguous values matching the chosen type).
  Slash dates resolve via `dateorder` (`auto`/`dmy`/`mdy`); genuinely ambiguous date columns
  stay `text` and are annotated — the value is never guessed.
- **Normalisation** (only where unambiguous): dates → ISO `YYYY-MM-DD`, emails lowercased,
  UK postcodes canonicalised (`sw1a1aa` → `SW1A 1AA`), URLs lowercased host + trailing dot
  trimmed, leading-zero values kept verbatim as `text`.
- **Dedupe**: exact and near-duplicate rows (case/whitespace-insensitive key). First occurrence
  kept; every removal appears in `dropped[]` with a reason. Nothing is silently dropped.
- **Invalid-row report**: per-row reason (which column failed validation and why).
- **Column hints**: force a column type; values that fail the forced type mark the row invalid
  rather than being rewritten.
- **Honest statuses**: report `status` is `ok | invalid | ambiguous`. `ambiguous` means a column
  could not be typed without guessing — the raw value is preserved.

### What it does NOT do

- No network calls at runtime — no fetch, no enrichment, no geocoding, no lookups.
- No enrichment of any kind: it will not append data it does not have.
- No fuzzy rewriting of values: normalisation is only the deterministic rules listed above;
  ambiguous values are kept as-is and annotated.
- No type guessing on ambiguous columns; no silent row drops (every drop has a reported reason).
- No persistence: one input in, one report out.

### Quickstart

```bash
cd /home/hermes/venture/csv-cleaner
node main.js --fixtures          # runs the fixture suite; prints "N/N passed", exit 0 on pass
node main.js input.json          # process { "csv": "..." } or { "rows": [...] }, prints report JSON
```

`input.json` example:

```json
{
  "csv": "Name, Email\n  Jon Smith ,jon@ex.com\n",
  "hints": {},
  "dateorder": "auto",
  "strictwidth": true
}
```

Object-array input is also accepted: `{"rows": [{"name": "Jon", "email": "jon@ex.com"}, …]}` is
serialised to CSV text (header = union of keys, first-seen order) before parsing, producing the
same report as the equivalent CSV string. `{"rows": []}` returns the documented `invalid` /
`empty input` report (no crash). Nested object/array values are not flattenable: they render as
empty cells and are counted in `summary.nested_object_cells`.

### Inputs / outputs

- `input_schema.json` — Apify-style input: `csv` (string) or rows, `hints`, `dateorder`,
  `strictwidth`. All property names lowercase with prefill defaults.
- `output_schema.json` — documents every field of the cleaning report
  (`status`, `reason`, `headers`, `columns[]`, `rows[]`, `dropped[]`, `summary`).

### Fixtures

- `fixtures/cases.json` — 48 hand-checked cases across 9 suites (type sniffing, header
  normalisation, CSV parse edges incl. BOM/CRLF, column resolution, exact + near dedupe,
  malformed rows, hint enforcement, line-string rows, object-array rows). `node main.js --fixtures` emits 48 dataset items.

### Positioning

P1 lead-hygiene companion rail: lead lists arrive as messy CSVs; P1 sells
freshness/verification — hygiene is the adjacent deterministic step. Differentiator:
bundled-offline execution + typed-schema machine-readable output (not a web-wrapper).

# Actor input Schema

## `csv` (type: `string`):

Raw CSV text (header row first). BOM, CRLF and quoted fields handled. Provide either csv or rows.

## `rows` (type: `array`):

Alternative to csv: array of {column: value} objects (or CSV line strings). Serialised to CSV text with a header from the union of keys before parsing — same report as the equivalent csv string. Empty array returns the documented empty-input error.

## `hints` (type: `object`):

Optional per-column forced types: { "<header>": "int|float|date|postcode|email|url|bool|text" }. Forced values that fail validation mark the row invalid; they are never silently rewritten.

## `dateorder` (type: `string`):

Order used for slash dates (09/05/2026). 'auto' resolves from values where one ordering is impossible; genuinely ambiguous columns stay text.

## `strictwidth` (type: `boolean`):

true: rows with more fields than the header are reported invalid. false: header is padded with col\_<n> columns so the extra data is kept.

## Actor input object example

```json
{
  "csv": "Name, Email, DOB, Postcode, active\n  Jon Smith ,jon@ex.com,09/09/2026,sw1a1aa,YES\njon  smith,jon@ex.com,09/09/2026,SW1A 1AA,yes\nbad email,jane@ex.com,13/09/2026,M1 1AE,no\n",
  "rows": [],
  "hints": {},
  "dateorder": "auto",
  "strictwidth": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

Fields: status (ok|invalid|ambiguous — ambiguous = a column could not be typed without guessing), reason (present when status != ok), malformed\_quotes (count of lines with unterminated quoting), headers (normalised header list), columns\[] ({header, raw\_header, type (int|float|date|postcode|email|url|bool|text), confidence (0-1 share of unambiguous values of the chosen type), non\_empty, note (e.g. 'type forced by hint', 'ambiguous date order — kept as text', 'N off-type value(s)')}), rows\[] (cleaned rows; int/float/bool validated and typed, dates as ISO YYYY-MM-DD, emails lowercased, UK postcodes canonicalised, URLs lowercased host + trimmed trailing dot, leading-zero values kept verbatim as text), dropped\[] ({row\_index (1-based data row), reason (per-column validation messages or near-duplicate note)}), summary ({rows\_in, rows\_out, duplicates\_removed, invalid\_rows, blank\_lines\_skipped, nested\_object\_cells (only when object-array input contained nested values that cannot flatten to CSV)}).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("draeg82/csv-cleaner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("draeg82/csv-cleaner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call draeg82/csv-cleaner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,draeg82/csv-cleaner"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ofeRuF1GVTbGwYmmB/builds/8SVimNTkgjovIbD24/openapi.json