catchall-verify

Email verification that gives a graded, calibrated judgement on catch-all domains instead of returning "unknown".

Between 15% and 28% of business domains accept mail for every address, real or not. Every verifier can tell you that a domain is catch-all; almost none will tell you what to do about the address. They hand back "unknown" — Emailable on 34.2% of addresses, NeverBounce on 50.2% in a 2026 benchmark — and the buyer pays for a shrug.

This tool returns keep / test / discard, a probability, and the reasoning behind it. Where it genuinely cannot know, it says so, and says why — which is different from, and more useful than, "unknown".

---

What it actually does

The parts every verifier has. Syntax against RFC 5321, MX and null-MX (RFC 7505), disposable domains, role addresses, typo'd provider domains, duplicates, and an SMTP conversation that probes RCPT TO and never sends a message.

The part that is the point. When a domain accepts everything, SMTP has told you nothing, so the verdict has to come from somewhere else. It comes from six places:

1. Does the local part derive from the name on the row? j.smith for John Smith does; john.smith for Maria Gonzalez does not. Aware of diacritics, transliteration (Koutný → koutny), truncation (kuniyu for Kuniyuki), initials, hyphenated surnames, middle names, and numeric disambiguators.

2. Does it follow the naming convention the rest of the list shows for that domain? This is the strongest computed signal and it is free: the customer already sent you twenty addresses at acme.com. If eighteen are first.last, the one that is xk29z is not a mailbox. Local parts are read ambiguously on purpose — smith could be a surname or s+mith — and the reading that makes the domain self-consistent wins.

3. Does one person appear under several spellings at one domain? j.smith, john.smith and jsmith at the same company means a guessing tool ran. At most one is real; all three are marked down.

4. Is it one edit from another address on the same list? That is a typo of a real address, not a second mailbox.

5. Does the employer named on the row match the domain? This is the only signal that reaches the question no structural check can: not "is this well formed" but "does this person work here". See the numbers below — it is the difference between 76% and 100% keep-precision.

6. What did the server leak despite accepting everything? Some catch-alls word their acceptance differently for a recipient they know; some take measurably longer; some backup relays reject what the primary accepts. All three are probed, all three are reported as weak evidence, and none is required.

Every signal feeds one calibrated model. A missing signal contributes zero — never a penalty — so a list without a name column is scored honestly rather than punished.

---

How good it is

Measured on a held-out test split of 319 addresses across 57 domains that the model was never fitted on, at 42% contamination. SMTP was disabled throughout, which reproduces the hardest case: a catch-all that accepts everything and leaks nothing.

Ranking quality (AUC)	0.815
Catch-all addresses given a verdict	42.9%
Correct when it gives one	85.4%
Precision of keep	92.9%
Precision of discard	77.6%
Genuine addresses lost to discard	4.8%
Calibration error	4.6%

The baseline every competing tool offers on these same 319 addresses is 0% resolved.

Calibration — a stated probability means what it says:

Said	Actually real
12%	18%
29%	25%
49%	49%
70%	77%
88%	93%

The company column changes the answer materially:

	Resolved	Correct	keep precision
Rows with a company column	41.8%	89.2%	100%
Rows without one	46.7%	74.3%	76.2%

Where the signal comes from, by contamination type:

Type	n	Correctly placed
Real addresses → keep	185	35.1% keep, 8% wrongly discarded
Machine-generated junk	18	83.3% discarded
Wrong spelling for that employer	42	31.0% discarded, 0% kept
Typo of a real address	17	29.4% discarded, 0% kept
Right person, wrong employer	57	33.3% discarded, 8.8% wrongly kept
Role addresses	17	94.1% flagged as role, not as a person

You choose where to sit. At the shipped operating point keep is 93% precise; raise the threshold to 0.90 and it is 100% precise on 16% of the real addresses instead of 35%.

---

What it does not do

Read this part before quoting a number to anyone.

• It cannot tell you whether someone works somewhere. A correctly-spelled address for a real person at a company they have never worked at is invisible to every structural signal — it is a well-formed address. This was the single largest error source in evaluation, and the company column is the only thing that touches it. A departed employee's old address is the same case and is equally invisible.
• It cannot confirm a catch-all mailbox exists. Nobody can, without sending mail. Every catch-all verdict here is a probability. Treating keep as "guaranteed deliverable" will produce bounces at roughly the rate the precision figure implies.
• It does not send email. The SMTP prober issues RCPT TO and stops. No DATA, no message body, ever. That also means post-acceptance bounces are invisible to it.
• It does not scrape anything. No web pages, no social networks, no third-party APIs, no logins. It reads the list you give it and asks DNS and the receiving mail server — the same two systems that would carry the mail.
• The measurement has known limits. The truth set is built from public commit metadata of open-source contributors: real people, real corporate domains, real naming conventions, but they use short handles (tj, ljs, axboe) far more than a sales contact list does. That makes the recall figure pessimistic for a typical B2B list and the convention signal harder than it would usually be. Three features that fitted well on this corpus but looked like artifacts of it (very_short_local, single_token, not_derived_partial) were removed after an ablation showed they were worth 0.003 AUC — not enough to justify shipping a bias that cannot be defended.
• wrong_spelling labels carry noise. Some employers really do alias j.smith to john.smith. Those rows are labelled fake and counted against the tool, so the true figure is somewhat better than reported. It is reported the pessimistic way on purpose.

---

Privacy: what is kept

Nothing. Addresses are read, processed in memory, and returned.

• No database, no cache file, no log of addresses. Nothing is written to disk by the engine at any point.
• Records are joined inside a single run by an index plus a random salt generated at the start of the call and discarded when it returns. There is no identifier that survives a run or links two runs.
• DNS results are cached per run, keyed by domain, never by address.
• On Apify, results go to the run's dataset because that is how the customer collects them; that dataset belongs to the customer's account and is theirs to delete. Set a retention period on it.
• The evaluation corpus contains real third-party addresses. It lives in eval/cache/, which is gitignored, and is never committed. Only aggregate metrics leave that directory.

This is a deliberate choice, and it costs something: a cross-run reputation database of which domains behave how would improve accuracy. It is not worth holding other people's addresses to get it.

---

Running it

npm install          # apify, for the platform wrapper only; src/ has no dependencies
npm test             # 28 tests, including a mock SMTP server

# a list
node bin/cli.js leads.csv --out verified.csv

# without SMTP: DNS and computed signals only, no port 25 needed
node bin/cli.js leads.csv --no-smtp

# keep a cleaner set at the cost of keeping fewer
node bin/cli.js leads.csv --keep 0.90 --discard 0.15

# enable the latency comparison on catch-alls (costs extra probes)
node bin/cli.js leads.csv --timing 5

Columns are detected by name: email / e-mail / mail, first_name + last_name or name, company / organization / employer. Header spelling and separators do not matter. Supply the name and company columns if you have them — the table above shows what they are worth.

As a library:

import { verifyList } from 'catchall-verify';

const { results, summary } = await verifyList([
  { email: 'j.smith@acme.com', first_name: 'John', last_name: 'Smith', company: 'Acme Ltd' },
], { smtp: { enabled: true } });

Each result carries action, probability, confidence, status, and an evidence array of plain-language reasons.

Port 25

SMTP verification needs outbound port 25, which most residential ISPs and many cloud providers block.

Apify blocks it too, and this was measured rather than assumed: from inside an Actor container, ports 25, 465 and 587 all fail with ETIMEDOUT in roughly 250 ms, while port 443 connects in 6 ms. That is a network-level block, not a slow server — which is what you would expect from a platform that does not want to become a spam source. So the Actor defaults SMTP off and says so in the input; turning it on there only produces a warning in the log.

This costs the product almost nothing, which is the point worth understanding. A catch-all domain is defined by SMTP telling you nothing, so on exactly the addresses this tool exists to judge, the SMTP layer was never going to contribute. Every number in the table above was measured with SMTP disabled. The SMTP prober earns its keep on non-catch-all domains — where it can flatly confirm or deny a mailbox — and for that you need to run the engine yourself, via bin/cli.js or as a library, somewhere port 25 is open.

---

Re-measuring and re-fitting

The model is not hand-tuned. Coefficients are fitted; thresholds are selected; both are reproducible.

npm run truthset   # harvest public commit metadata -> eval/cache/  (gitignored)
npm run fit        # fit coefficients + calibration + thresholds -> data/weights.json
npm run eval       # evaluate on the held-out split -> eval/RESULTS.txt

Method, and why:

• Sign-constrained logistic regression. On a few hundred rows a free fit will invert a pair of collinear signals and conclude that matching a domain's naming convention makes an address less likely real. The direction of each signal is known in advance and fixed; only the magnitude is learned. 18 of 27 features are constrained.
• Grouped 5-fold cross-validation, folded by domain. Cohort signals make rows at one domain dependent on each other, so a row-wise split would leak.
• Thresholds and calibration chosen out-of-fold, never in-sample. An earlier version chose them in-sample and promised 85% discard precision while delivering 64% on unseen domains. That is what in-sample selection does.
• Asymmetric constraints, because the two mistakes do not cost the same. Keeping a fake address produces a bounce, and bounces damage a sending domain's reputation — so keep is held to a hard 90% precision floor. Discarding a real address loses one prospect out of thousands — so the discard side is constrained by the share of genuine addresses it throws away (≤5%).
• The test split is never touched during fitting.

npm run dict rebuilds the bundled name model. You should not need to: the bundled data is a given-name list and a character model of human names. Neither goes stale.

---

Maintenance

Designed to survive a maintainer being at sea for a fortnight.

Nothing needs regular updating. There is no dependency tree in the engine — src/ uses only Node built-ins. The bundled data is a given-name list, a role-address list, a disposable-domain list and MX fingerprints: all slow-moving, and none of them load-bearing (the machine-generated-address detector is statistical and uses no list at all, so it cannot rot).

The one real drift risk is that Microsoft and Google periodically change how their servers answer probes. That affects the SMTP layer, not the catch-all scoring — and when it happens, --no-smtp still produces the verdict this tool exists for. Budget one afternoon a year.

Apify's own obligations are the tighter constraint: 14 days to respond to a user report, 3 working days to respond to Apify. An Actor that only computes on submitted input has nothing to break while unattended, so the automated health check will not fail on its own.

---

Publishing to Apify

Everything is in place. The steps, in order:

1. npm install -g apify-cli

2. apify login — needs an Apify account; the free plan is enough to publish.

3. apify push from this directory. It reads .actor/actor.json, builds the Dockerfile and uploads. First build takes a few minutes.

4. Test the build in Apify Console → the Actor → Input tab. actor/INPUT.json holds a working example.

5. Publication tab → set title, description, categories (Lead generation, Developer tools), and a README (this file works).

6. Pricing → Pay per event, and define two events with exactly these names, because actor/main.js charges against them:

   • address-verified — charged once per address
   • catch-all-resolved — charged once per catch-all given a verdict instead of "unknown"

   Suggested: $1.00–1.50 per 1,000 on address-verified, and either nothing or a small premium on catch-all-resolved. The second event is the honest one to charge for — it only fires when the tool did the thing the competition does not. Measured data says Actors above $15/1,000 results earn ~22× those under $1/1,000 despite a seventh of the traffic, so pricing above the $0.36–0.89/1,000 floor is the right call.

7. Set the Actor to Public.

8. Payouts → identity verification (passport or driving licence photo) plus PayPal, Wise or bank details. Threshold $20 via PayPal/Wise, $100 via bank. Invoice on the 11th, auto-approved on the 14th. You keep 80%; Apify's margin is 20% and is visible in their API as apifyMarginPercentage. If your price does not cover platform costs your profit is floored at zero — you are never billed.

Steps 1–3 are one command each and can be done in a single sitting. Steps 5–8 are the Console and need decisions, not code.

---

Layout

src/                 the engine — no dependencies, no platform knowledge
  index.js           orchestration: parse -> DNS -> SMTP (batched per domain) -> cohort -> judge
  syntax.js          RFC parsing, canonical form, role/disposable/typo
  dns.js             MX/SPF/DMARC, provider fingerprinting, absent-vs-unanswerable
  smtp.js            RCPT probing, catch-all detection, reply/timing/secondary-MX divergence
  score.js           features, sign priors, calibrated probability, verdict, explanations
  report.js          CSV/JSONL in and out
  signals/           pattern (name derivation), nameness (character model),
                     cohort (within-domain), company (employer-to-domain)
data/                given names, roles, disposables, MX fingerprints, fitted weights
eval/                harvest, truth-set construction, fitting, evaluation
actor/ .actor/       the Apify wrapper and its schemas
test/                28 tests, including a mock SMTP server
bin/cli.js           command line

Licence

MIT.