Contact & Company Deduplicator

Upload one messy contact or company list — exported from a CRM, a spreadsheet, a form tool, wherever — and get back a clean list with the duplicates merged, ready to re-import. No shared ID required: it figures out that "Jennifer Adams / Acme Corporation" and "J. Adams / ACME, Inc." are the same person from the data itself.

This is the simpler, single-list sibling of the Entity Resolution Engine actor. Same matching engine underneath (fuzzy name/company matching, blocking, probabilistic scoring, transitive clustering, survivorship), but zero-config by default and with output shaped specifically for CRM cleanup instead of general entity resolution.

What it does

1. Auto-detects which columns are name, email, phone, company and address (or you can specify them yourself).
2. Groups obviously-unrelated rows apart first ("blocking"), so a 20,000-row list doesn't need 200 million comparisons.
3. Scores every remaining pair on weighted field similarity — an email match counts for much more than two people happening to share a surname.
4. Merges everything that scores above the match threshold, even transitively (AB and BC means A, B and C become one contact), and reports anything in between for you to decide.
5. Builds one "golden" record per real contact, picking the best value for each field when members disagree.

Input

{
  "contacts": [
    { "name": "Jennifer Adams", "company": "Acme Corporation", "email": "jen@acme.com", "phone": "415-555-0100" },
    { "name": "Jen Adams", "company": "Acme Corp.", "email": "jen@acme.com", "phone": "" }
  ],
  "matchThreshold": 85,
  "reviewThreshold": 70
}

Only contacts is required. Everything else has a sensible default: 85/70 thresholds, "most complete record wins" on conflicts, duplicates-to-review and a merge audit log both included.

Output

One dataset, rows tagged by type:

type	what it is
contact	The clean, deduplicated list — always emitted. One row per real contact, _dedupGroupSize/_dedupMerged/_dedupConfidence columns are metadata, safe to drop before re-importing.
duplicateGroup	A group of rows that got merged, shown for a sanity check.
reviewPair	A pair that scored too close to call — not merged, flagged for a human.
mergeLog	One entry per merged group: which fields agreed (matchedOn), which fields disagreed (conflicts, with every value seen and which one was kept), and how many records were folded in.

The key-value store's SUMMARY reports estimatedRecordsSaved (rows eliminated), reductionPercent, and conflictsByField — a count of how often each column disagreed across merged groups, useful for spotting a chronically dirty field (e.g. phone numbers entered in ten different formats).

What it will NOT do

• It will not merge two different people who happen to share a company and surname (e.g. "John Smith" and "Jane Smith" at the same firm) — a shared surname or company alone never crosses the match threshold on its own.
• It does not call out to any external lookup or enrichment service — matching is based purely on the fields you give it. Garbage columns in means a weaker signal, not an error.
• It caps at 200,000 rows per run and skips oversized "blocks" (e.g. a thousand contacts sharing one generic info@ domain) to avoid a runaway comparison count — those rows still get compared under every other field, just not against each other via that one key.
• reviewThreshold must not exceed matchThreshold — the run fails fast rather than merging everything.

Who uses it

Sales ops cleaning a CRM before a migration · anyone who just merged two contact lists after an acquisition · agencies handing back a de-duplicated lead list to a client · marketers about to send an email blast who don't want the same person to get it three times.