Email Lead List Quality & Deduplication Audit

Audit supplied lead lists before CRM import

Email Lead List Quality & Deduplication Audit turns an existing Apify dataset or inline JSON rows into a deterministic quality report. It is designed for the practical step between collecting leads and importing them into a CRM, engagement system, spreadsheet, or agent workflow. The Actor finds exact normalized-email duplicates, checks email syntax, compares email hosts with supplied company websites, measures selected contact-field coverage, labels common role and free-provider addresses, and preserves caller-supplied verification evidence.

It does not scrape websites, call another Actor, send email, perform DNS or MX queries, open SMTP connections, verify a mailbox, infer consent, or remove records. There is no LLM. A ready_structural result means only that the observed row satisfies your selected field contract; it is not evidence of deliverability, ownership, permission to contact, commercial intent, or future performance.

What makes the output useful

• Read up to 100,000 rows from a selected Apify dataset, or paste up to 10,000 inline rows.
• Recognize current Compass business-leads fields, Apollo-style nested organization fields, common generic exports, and a strict custom dotted-path map.
• Normalize email case for exact duplicate grouping without changing the source dataset.
• Report syntax failures, role-address and common free-provider classifications, exact/subdomain company-host relationships, and selected field coverage.
• Preserve allowlisted upstream verification results as labeled caller-supplied evidence. Unexpected values become unrecognized instead of silently acquiring new meaning.
• Emit bounded scorecards and duplicate groups plus complete portfolio counts, cap metadata, and warnings for agent-safe downstream decisions.
• Keep human names, job-title values, LinkedIn values, and company/account identifiers out of output. Contact values are also off by default.

The Actor recognizes aliases, not provenance. Selecting compass_business_leads or apollo_style labels the expected source contract but does not prove where a row came from, endorse a provider, or establish a license to use the data. This Actor is not affiliated with, endorsed by, or sponsored by Apify marketplace data providers.

Input

Choose exactly one practical source:

1. source_dataset_id: select an existing dataset with read-only resource permission. This takes precedence over inline rows.
2. leads: paste lead objects directly for a quick audit.

The default required_fields are email, company_name, and website. You can instead require any distinct selection of email, company name, website, phone, name presence, job-title presence, LinkedIn presence, or company location presence. Percentages are direct unweighted counts across the chosen fields.

Use field_map when your export uses different keys. Each mapping is a safe dotted path, for example:

{
  "source_dataset_id": "YOUR_DATASET_ID",
  "source_contract": "custom",
  "field_map": {
    "emails": "contact.work_email",
    "company_name": "account.display_name",
    "website": "account.domain",
    "phones": "account.phones",
    "verification_result": "contact.email_check.result"
  },
  "required_fields": ["email", "company_name", "website", "phone"],
  "max_rows": 25000,
  "max_source_bytes": 33554432,
  "max_source_row_bytes": 1048576,
  "include_company_names": true,
  "include_contact_values": false
}

Arrays or strings are accepted for email and phone fields. max_scorecard_rows, max_duplicate_groups, and max_duplicate_group_size bound detailed output while summary counts continue to cover every analyzed row. Set as_of to an ISO 8601 timestamp when reproducible fixtures require a stable generated_at value.

Both source modes use the same row, aggregate compact-JSON byte, and per-row byte ceilings. Selected Datasets are read in pages with a plus-one row probe. The Actor stops before a row that crosses a byte ceiling and reports source_json_bytes, transport completeness, exact cap flags, and closed truncation reasons in run_meta; it never partially parses an oversized row.

Output records

The default dataset contains four explicit record_kind values in a billing-safe order:

• accepted non-invalid lead_scorecard rows form one exact leading paid prefix;
• one free portfolio_summary is recomputed over that delivered prefix plus free invalid rows;
• free lead_scorecard rows whose status is invalid follow the summary;
• free duplicate_email_group records follow only when every listed source row is in the delivered cohort; and
• one free run_meta records the selected contract, source receipt, effective output caps, privacy switches, billing counts, and immutable-plan fingerprint.

The same order is used under FREE pricing. source_rows_read remains the bounded ingestion receipt, while readiness, field coverage, syntax, host-relation, upstream-label, and duplicate metrics describe only delivered scorecards. paid_lead_rows_planned, paid_lead_rows_delivered, paid_lead_rows_charged, paid_lead_rows_omitted, and spending_limit_reached expose the exact boundary. Paid rows contain only row-scoped warnings and never a full omitted-cohort summary.

Each scorecard has one precedence-ordered quality_status: invalid, duplicate_email, review_upstream_email, domain_mismatch, missing_required_fields, or ready_structural. Precedence makes the main review queue stable without hiding secondary evidence; the remaining evidence fields stay machine-readable.

With include_contact_values: false, scorecards contain counts and labels but no emails, normalized phones, or website hosts. With include_contact_values: true, only those supplied contact values are copied. Human contact names, job-title text, profile URLs or IDs, and stable account identifiers are never emitted in either mode. Company names can be controlled separately with include_company_names.

Exact duplicate and domain rules

Duplicate groups use case-folded, syntax-valid email equality. This is reliable evidence that two supplied rows share the same normalized address, but it is not proof that the people, accounts, or companies are duplicates. The Actor never performs fuzzy identity matching, pair expansion, automatic merging, deletion, or selection.

Company-domain checks deliberately avoid registrable-domain guessing. An email host is a match only when it equals the supplied website host or one is an exact subdomain of the other. A recognized consumer mailbox host is free_provider; a different host is mismatch; multiple classifications are mixed. These local labels are review aids, not risk or quality verdicts.

Phone normalization removes punctuation but does not claim E.164 validity or reachability. A leading calling code 1 is removed only when an observed or custom-mapped country explicitly identifies the United States or Canada. With missing or other country evidence, the complete digit sequence is retained rather than silently assuming NANP.

Cost and operation

Runtime is primarily linear in the number of supplied rows and normalized email values. Exact-email groups use dictionary buckets rather than all-pairs comparison. A 128 MB minimum is suitable for small fixtures; increase memory for larger datasets and measure Cloud cost for your own input shape.

The active PAY_PER_EVENT contract is $0.00025 per lead-row-cleaned event ($0.25 per 1,000 accepted rows) plus the platform-generated apify-actor-start event at $0.00005. The live Monetization panel says User pays platform usage costs: No, so normal run platform usage is included in these event prices. A paid unit is exactly one non-invalid lead scorecard durably visible in the leading Dataset prefix. Invalid rows, the effective summary, cohort-scoped duplicate evidence, and the run receipt have no custom result-event fee; there is no apify-default-dataset-item event. Post-run Dataset access can still incur normal storage-operation charges under Apify's Store billing rules.

$0.00025 is intentionally conservative. Public mailbox verifiers with meaningful use are currently around $0.0006-$0.001 per email, while this Actor performs no DNS, SMTP, or mailbox verification. A lightly used B2B list cleaner is around $0.002 per row. The lower launch price reflects the narrower structural result. Code rejects a lead-row-cleaned price above $0.00025, a start price above $0.00005, an unknown configured/positively charged event, or a positive Dataset-item price.

At the active $0.001 minimum maximum charge, a $0.00005 start leaves capacity for three $0.00025 accepted rows. Each caller-selected budget at or above that floor yields its exact affordable prefix. Each new prefix is saved and read back before a direct idempotent charge. The complete derived result is first frozen in chunked, SHA-256-verified run KVS records bound to parsed input and run identity. Restarts reconcile stored rows, live and charging-manager counts, response loss, stale counts, and spending capacity; once free evidence begins, the paid prefix is final.

For local verification:

python3.11 -m venv .venv
.venv/bin/python -m pip install -r requirements-dev.txt -c requirements.lock
.venv/bin/python -m pytest -q
.venv/bin/python -m src

Limitations and responsible use

Use only datasets you are authorized to process. Review source terms, privacy obligations, suppression lists, consent requirements, and applicable anti-spam rules before any contact campaign. Syntax-valid does not mean deliverable. A company-domain match does not mean the address belongs to that company. A role or free-provider address is not inherently bad. Caller-supplied verification evidence may be stale, incomplete, or wrong.

The report covers supplied_lead_rows_only; it makes no claim about all leads, a source provider's entire database, or unobserved fields. Caps and truncation are explicit. If a source shape changes, use a custom field map and test a bounded private fixture before scheduling production runs.

The repository includes a sealed 200-row synthetic benchmark across auto-detected, declared Compass-style, declared Apollo-style, and custom-mapped contracts. It uses .example domains and 555 numbers, independently labels statuses and exact duplicate groups, and contains no redistributed marketplace or real lead data. This structural corpus does not replace current owner-authorized source-shape acceptance before release.

Support

When reporting a problem, include the Actor run ID, selected source contract, redacted input shape, record counts, expected status, actual status, and whether any cap was reached. Do not send credentials, private datasets, raw personal records, API tokens, or unredacted lead values in a public issue.