# Lead List Cleaner: Dedupe & Tidy CRM CSVs (`silent_jubilance/lead-list-cleaner`) Actor

Clean a lead list you already have before CRM import: remove duplicates, trim whitespace, lowercase emails, optionally format US/Canada phones, and flag invalid, missing or personal-domain emails. Input an Apify dataset or CSV URL. No scraping, no enrichment.

- **URL**: https://apify.com/silent_jubilance/lead-list-cleaner.md
- **Developed by:** [Sales List Rescue](https://apify.com/silent_jubilance) (community)
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 68.4% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Lead List Cleaner: Dedupe & Tidy CRM CSVs

▶ Watch it in 30 seconds: [YouTube Short](https://youtube.com/shorts/6v2df6xPFbA)

Clean a lead list **you already have** before you import it into HubSpot, Salesforce, Pipedrive or any other CRM. Give it an Apify dataset (for example the output of a scraper run you own) or a link to a CSV, and get back a de-duplicated, tidied table, a per-row list of flags and edits, and a run summary.

Built by Sales List Rescue on top of our MIT-licensed `crm-csv-preflight` checker.

### What it does

- **Removes duplicates** on the columns you choose (default: email). The comparison ignores case and surrounding spaces, and the first occurrence is kept. You can also keep duplicates and only flag them.
- **Standardizes formatting**: trims and collapses whitespace in every cell and lowercases email columns. Two optional steps: format US/Canada phone numbers as `+1XXXXXXXXXX`, and title-case names that are ALL CAPS or all lowercase.
- **Flags issues per row**: `missing_email`, `invalid_email` (format only), `personal_domain` (gmail.com, yahoo.com, etc.), `bad_phone` (not 10 or 11 digits), `missing_company`, `ragged_row` (wrong number of cells), and `duplicate_of_row_N`.
- **Records every edit**, so you can audit each change (`column`, `before`, `after`).

### What it does NOT do

- **No scraping, no enrichment, no lookups.** It never adds contacts, never fills in blank cells, and never fetches anything except the CSV URL you give it.
- **No deliverability or bounce checks.** `invalid_email` is a format check only. A well-formed address can still bounce, and we make no promises about deliverability.
- It does not merge near-duplicates (for example "Acme Inc" vs "Acme"). Only exact matches on your keys, after cleaning, count as duplicates.
- It does not validate phone numbers outside the US and Canada. The phone check simply counts digits.

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `datasetId` | dataset picker | n/a | An Apify dataset. Read-only access is requested. Use this **or** `csvUrl`. |
| `csvUrl` | string | n/a | A direct, public `http(s)` link to a CSV. Comma, semicolon, tab or pipe delimiters; UTF-8, cp1252 or latin-1. Use this **or** `datasetId`. |
| `dedupeKeys` | string\[] | `["email"]` | An exact column name, or a field type: `email`, `phone`, `company`, `full_name`, `website`, `title`, `linkedin`. When you list several keys, a row counts as a duplicate only if all of them match. An empty list turns de-duplication off. |
| `removeDuplicates` | boolean | `true` | When off, duplicates are kept and flagged. |
| `trimWhitespace` | boolean | `true` | |
| `lowercaseEmails` | boolean | `true` | |
| `normalizePhones` | boolean | `false` | US/Canada numbers only. Anything else is left unchanged. |
| `titleCaseNames` | boolean | `false` | Only changes words that are all upper- or all lowercase. Mixed case such as "McDonald" is left alone. |
| `flagIssues` | boolean | `true` | |
| `issuesColumnInCsv` | boolean | `false` | Adds a `preflight_issues` column to `CLEANED_CSV` with the plain-English reason for each flag. When off, the CSV has your columns only. |

You must provide exactly one of `datasetId` or `csvUrl`. If you provide both or neither, the run fails with a clear message.

Example:

```json
{
  "csvUrl": "https://docs.google.com/spreadsheets/d/e/EXAMPLE/pub?output=csv",
  "dedupeKeys": ["email"],
  "normalizePhones": true
}
```

### Output

- **Dataset**: one item for each row that was kept. It contains your original columns with cleaned values, as strings (nested dataset values are stored as JSON text). It also has `preflightRow` (the source row number), `preflightFlags` and `preflightChanges`. The **Flags & changes** view shows the audit columns; **All fields** or a CSV/Excel export gives you the full cleaned table.
- **`preflightIssues`** (dataset field, shown in the **Flags & changes** view): a plain-English reason for each flag on the row, joined with `; `, such as `Email 'hello@fictional-fitness' is not a valid email format (not auto-fixed)`; empty when the row has no flags.
- **Key-value store `SUMMARY`**: `inputRows`, `blankRowsSkipped`, `outputRows`, `duplicatesFound`, `duplicatesRemoved`, `cellsChanged`, `rowsChanged`, `rowsFlagged`, `flagCounts`, `dedupeColumns`, `unresolvedDedupeKeys`, `columnMapping`, `options`, plus the `crm-csv-preflight` report before and after cleaning.
- **Key-value store `CLEANED_CSV`**: the kept rows as a UTF-8 CSV with your columns only, ready to import.

Example output item (synthetic data):

```json
{
  "Company": "Example Robotics LLC",
  "Contact Name": "Jane Doe",
  "Email": "jane.doe@example-robotics.example",
  "Phone": "+14155550198",
  "preflightRow": 2,
  "preflightFlags": [],
  "preflightChanges": [
    {"column": "Email", "before": " Jane.Doe@Example-Robotics.example ", "after": "jane.doe@example-robotics.example"},
    {"column": "Phone", "before": "(415) 555-0198", "after": "+14155550198"}
  ]
}
```

### Example tasks

- [Dedupe a messy CRM export by email + phone](https://apify.com/silent_jubilance/lead-list-cleaner/examples/dedupe-crm-export-email-phone): 20 rows in, 16 out, 4 duplicates removed, 5 rows flagged. Sample file: https://pureservices.store/samples/messy-crm-20.csv
- [Clean & dedupe a Google Maps export (title → company)](https://apify.com/silent_jubilance/lead-list-cleaner/examples/clean-dedupe-google-maps-export): 20 rows in, 16 out, 4 duplicates removed by `placeId`, 5 rows flagged, and the Maps `title` mapped to company. Sample file: https://pureservices.store/samples/maps-export-20.csv

Both sample files are fictitious demo data.

### Limits

- Up to 50 MB and 100,000 rows per run. Split larger files. Default memory is 1 GB, which is enough for typical lists. For files near the 50 MB cap, start the run with 2 GB, which is the Actor's maximum.
- Columns are detected from header names (for example "E-mail", "Mobile" or "Organization"). "Business Name", "Company Name" and Google Maps' `title` field (when the input has Maps fields such as `placeId` or `categoryName`) are treated as the company, not a person's name. Google Maps' `categoryName` is read as a category, so name title-casing never changes it, and on Google Maps data a `name` dedupe key no longer matches `categoryName`: to dedupe on business name pass `title`, or pass `categoryName` explicitly to dedupe by category. The detected mapping is in `columnMapping` in the `SUMMARY` record; if a column has an unusual name, put the exact name in `dedupeKeys`.
- Column order is kept. Fully blank rows are skipped. Extra cells on a ragged row go into an `overflow` column, so nothing is dropped.
- Permissions: the Actor runs with **limited permissions**. It reads only the dataset you pick and writes only to its own run storage.

### Want a hand?

- Free in-browser CSV preflight check: https://pureservices.store/tools/csv-preflight?utm_source=apify\&utm_medium=actor\&utm_campaign=path21
- Prefer a done-for-you clean? Our $25 Quick Clean is available at [Sales List Rescue](https://pureservices.store/?utm_source=apify\&utm_medium=actor\&utm_campaign=path21).

Each run's log ends with the free-check link above.

### Local development

```bash
pip install -r requirements.txt pytest
pytest tests                        # core cleaner tests; the Apify SDK is not needed
scripts/smoke_local.sh python3      # full Actor run against local storage (no Apify account)
```

# Actor input Schema

## `datasetId` (type: `string`):

Apify dataset to clean (read-only access is requested). Use this OR CSV URL, not both.

## `csvUrl` (type: `string`):

Direct http(s) link to a CSV file, e.g. a Google Sheets 'Publish to web → CSV' link. Comma, semicolon, tab and pipe delimiters; UTF-8, cp1252 or latin-1. Max 50 MB / 100,000 rows. Use this OR Source dataset, not both.

## `dedupeKeys` (type: `array`):

Columns that define a duplicate. Use an exact column name (case-insensitive) or a field type: email, phone, company, full_name, website, title, linkedin. Several keys = all must match. Comparison ignores case and surrounding spaces; rows where every key is blank are never treated as duplicates. Empty list = no de-duplication. The first occurrence is kept.

## `removeDuplicates` (type: `boolean`):

On: later duplicates are dropped from the output. Off: they are kept and flagged duplicate_of_row_N.

## `trimWhitespace` (type: `boolean`):

Strip leading/trailing spaces and collapse repeated spaces in every cell.

## `lowercaseEmails` (type: `boolean`):

Lowercase every column detected as an email column.

## `normalizePhones` (type: `boolean`):

Rewrite 10-digit numbers (or 11 digits starting with 1) in phone columns as +1XXXXXXXXXX. Other numbers are left unchanged. Off by default.

## `titleCaseNames` (type: `boolean`):

In name columns, title-case words that are entirely upper- or lower-case ("jane DOE" → "Jane Doe"). Mixed-case words such as McDonald are left alone. Off by default.

## `flagIssues` (type: `boolean`):

Add per-row flags: missing_email, invalid_email (format check only, not a deliverability check), personal_domain, bad_phone, missing_company, ragged_row, duplicate_of_row_N.

## `issuesColumnInCsv` (type: `boolean`):

Append a preflight_issues column to CLEANED_CSV with the plain-English reason for each flag, e.g. "Email 'sam@@x.example' is not a valid email format (not auto-fixed)". Off by default, so the CSV keeps exactly your columns. The dataset always has the preflightIssues field.

## Actor input object example

```json
{
  "csvUrl": "https://pureservices.store/samples/lead-list-cleaner-sample.csv",
  "dedupeKeys": [
    "email"
  ],
  "removeDuplicates": true,
  "trimWhitespace": true,
  "lowercaseEmails": true,
  "normalizePhones": false,
  "titleCaseNames": false,
  "flagIssues": true,
  "issuesColumnInCsv": false
}
```

# Actor output Schema

## `cleanedRows` (type: `string`):

No description

## `cleanedCsv` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "csvUrl": "https://pureservices.store/samples/lead-list-cleaner-sample.csv",
    "dedupeKeys": [
        "email"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("silent_jubilance/lead-list-cleaner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "csvUrl": "https://pureservices.store/samples/lead-list-cleaner-sample.csv",
    "dedupeKeys": ["email"],
}

# Run the Actor and wait for it to finish
run = client.actor("silent_jubilance/lead-list-cleaner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "csvUrl": "https://pureservices.store/samples/lead-list-cleaner-sample.csv",
  "dedupeKeys": [
    "email"
  ]
}' |
apify call silent_jubilance/lead-list-cleaner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,silent_jubilance/lead-list-cleaner"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ti8DKQ3DgBiigwGiR/builds/8kHeyH5jiHDGQUoWt/openapi.json