Lead List Cleaner: Dedupe & Tidy CRM CSVs

▶ Watch it in 30 seconds: YouTube Short

Clean a lead list you already have before you import it into HubSpot, Salesforce, Pipedrive or any other CRM. Give it an Apify dataset (for example the output of a scraper run you own) or a link to a CSV, and get back a de-duplicated, tidied table, a per-row list of flags and edits, and a run summary.

Built by Sales List Rescue on top of our MIT-licensed crm-csv-preflight checker.

What it does

• Removes duplicates on the columns you choose (default: email). The comparison ignores case and surrounding spaces, and the first occurrence is kept. You can also keep duplicates and only flag them.
• Standardizes formatting: trims and collapses whitespace in every cell and lowercases email columns. Two optional steps: format US/Canada phone numbers as +1XXXXXXXXXX, and title-case names that are ALL CAPS or all lowercase.
• Flags issues per row: missing_email, invalid_email (format only), personal_domain (gmail.com, yahoo.com, etc.), bad_phone (not 10 or 11 digits), missing_company, ragged_row (wrong number of cells), and duplicate_of_row_N.
• Records every edit, so you can audit each change (column, before, after).

What it does NOT do

• No scraping, no enrichment, no lookups. It never adds contacts, never fills in blank cells, and never fetches anything except the CSV URL you give it.
• No deliverability or bounce checks. invalid_email is a format check only. A well-formed address can still bounce, and we make no promises about deliverability.
• It does not merge near-duplicates (for example "Acme Inc" vs "Acme"). Only exact matches on your keys, after cleaning, count as duplicates.
• It does not validate phone numbers outside the US and Canada. The phone check simply counts digits.

Input

Field	Type	Default	Notes
datasetId	dataset picker	n/a	An Apify dataset. Read-only access is requested. Use this or csvUrl.
csvUrl	string	n/a	A direct, public http(s) link to a CSV. Comma, semicolon, tab or pipe delimiters; UTF-8, cp1252 or latin-1. Use this or datasetId.
dedupeKeys	string[]	["email"]	An exact column name, or a field type: email, phone, company, full_name, website, title, linkedin. When you list several keys, a row counts as a duplicate only if all of them match. An empty list turns de-duplication off.
removeDuplicates	boolean	true	When off, duplicates are kept and flagged.
trimWhitespace	boolean	true
lowercaseEmails	boolean	true
normalizePhones	boolean	false	US/Canada numbers only. Anything else is left unchanged.
titleCaseNames	boolean	false	Only changes words that are all upper- or all lowercase. Mixed case such as "McDonald" is left alone.
flagIssues	boolean	true
issuesColumnInCsv	boolean	false	Adds a preflight_issues column to CLEANED_CSV with the plain-English reason for each flag. When off, the CSV has your columns only.

You must provide exactly one of datasetId or csvUrl. If you provide both or neither, the run fails with a clear message.

Example:

{
  "csvUrl": "https://docs.google.com/spreadsheets/d/e/EXAMPLE/pub?output=csv",
  "dedupeKeys": ["email"],
  "normalizePhones": true
}

Output

• Dataset: one item for each row that was kept. It contains your original columns with cleaned values, as strings (nested dataset values are stored as JSON text). It also has preflightRow (the source row number), preflightFlags and preflightChanges. The Flags & changes view shows the audit columns; All fields or a CSV/Excel export gives you the full cleaned table.
• preflightIssues (dataset field, shown in the Flags & changes view): a plain-English reason for each flag on the row, joined with ; , such as Email 'hello@fictional-fitness' is not a valid email format (not auto-fixed); empty when the row has no flags.
• Key-value store SUMMARY: inputRows, blankRowsSkipped, outputRows, duplicatesFound, duplicatesRemoved, cellsChanged, rowsChanged, rowsFlagged, flagCounts, dedupeColumns, unresolvedDedupeKeys, columnMapping, options, plus the crm-csv-preflight report before and after cleaning.
• Key-value store CLEANED_CSV: the kept rows as a UTF-8 CSV with your columns only, ready to import.

Example output item (synthetic data):

{
  "Company": "Example Robotics LLC",
  "Contact Name": "Jane Doe",
  "Email": "jane.doe@example-robotics.example",
  "Phone": "+14155550198",
  "preflightRow": 2,
  "preflightFlags": [],
  "preflightChanges": [
    {"column": "Email", "before": " Jane.Doe@Example-Robotics.example ", "after": "jane.doe@example-robotics.example"},
    {"column": "Phone", "before": "(415) 555-0198", "after": "+14155550198"}
  ]
}

Example tasks

• Dedupe a messy CRM export by email + phone: 20 rows in, 16 out, 4 duplicates removed, 5 rows flagged. Sample file: https://pureservices.store/samples/messy-crm-20.csv
• Clean & dedupe a Google Maps export (title → company): 20 rows in, 16 out, 4 duplicates removed by placeId, 5 rows flagged, and the Maps title mapped to company. Sample file: https://pureservices.store/samples/maps-export-20.csv

Both sample files are fictitious demo data.

Limits

• Up to 50 MB and 100,000 rows per run. Split larger files. Default memory is 1 GB, which is enough for typical lists. For files near the 50 MB cap, start the run with 2 GB, which is the Actor's maximum.
• Columns are detected from header names (for example "E-mail", "Mobile" or "Organization"). "Business Name", "Company Name" and Google Maps' title field (when the input has Maps fields such as placeId or categoryName) are treated as the company, not a person's name. Google Maps' categoryName is read as a category, so name title-casing never changes it, and on Google Maps data a name dedupe key no longer matches categoryName: to dedupe on business name pass title, or pass categoryName explicitly to dedupe by category. The detected mapping is in columnMapping in the SUMMARY record; if a column has an unusual name, put the exact name in dedupeKeys.
• Column order is kept. Fully blank rows are skipped. Extra cells on a ragged row go into an overflow column, so nothing is dropped.
• Permissions: the Actor runs with limited permissions. It reads only the dataset you pick and writes only to its own run storage.

Want a hand?

• Free in-browser CSV preflight check: https://pureservices.store/tools/csv-preflight?utm_source=apify&utm_medium=actor&utm_campaign=path21
• Prefer a done-for-you clean? Our $25 Quick Clean is available at Sales List Rescue.

Each run's log ends with the free-check link above.

Local development

pip install -r requirements.txt pytest
pytest tests                        # core cleaner tests; the Apify SDK is not needed
scripts/smoke_local.sh python3      # full Actor run against local storage (no Apify account)