Bulk Domain Intel: WHOIS/RDAP, DNS, Email Security & SSL

Paste a list of domains, URLs or email addresses. You get one clean row per domain with:

• Ownership & expiry (RDAP): registrar, creation and expiry dates, EPP status, DNSSEC.
• DNS: A, AAAA, MX, NS, TXT and CAA records, plus the DNS provider.
• Email security: mailbox provider, SPF, DMARC, common DKIM selectors and an A–F grade.
• SSL certificate: issuer, expiry, hostname coverage and trust chain.
• Website: status, redirects, HSTS, hosting/CDN and page title.
• Evidence-backed flags: every flag says why it was raised and how confident it is.

It uses only official, open data: the registries' RDAP servers (via the IANA bootstrap), public DNS, and direct TLS/HTTP connections. No login, no API keys, no scraping of third-party WHOIS sites.

How fast is it / how big can I go?

The full preset checks about 2–3 domains per second (1,000 domains ≈ 7 min; 5,000 ≈ 30 min). The ownership preset on .com/.net does about 10 per second. email_security does about 25 per second (10,000 ≈ 6 min). Runs over about 20,000 domains with the full preset need a longer timeout, or should be split into batches.

Registry limits we respect: .org's registry (PIR) blocks bulk lookups for an hour after a few hundred requests, so check at most about 250 .org domains per run. Any extra come back as rate_limited_retry_later (free) — re-run them after rdapRetryAfter. .com/.net and .uk handle thousands. .au lookups are off by default (auDA policy); polite mode is capped at 10 per run.

Registration dates aren't available for every TLD. .de, .io, .ru, .jp, .cn, .it and some other ccTLDs publish no RDAP. You still get DNS, email security, SSL and website checks for them (rdapStatus: unsupported_tld, not charged). We deliberately don't fall back to port-43 WHOIS: those registries' WHOIS terms forbid high-volume automated querying.

Your spend limit stops the run early. On the free plan, Apify caps a run at your remaining credit (about 1,800 full-preset domains). Set "Maximum cost per run" or pick a cheaper preset for big lists. When the limit is reached, the run status says "Stopped at your max cost per run (US$X): N of M domains done", every unchecked domain gets a free rowStatus: not_processed row, and the REMAINING record in the key-value store lists them so you can re-run just those. maxDomains caps a run up front. At the start, the log shows an ETA and (on pay-per-event pricing) the estimated cost.

Service	Safe batch per run (measured on Apify)
Verisign RDAP (.com/.net)	≥ 1,500
PIR RDAP (.org)	≤ 250 (built-in cap; the rest are free rate_limited_retry_later rows)
Nominet RDAP (.uk)	≥ 500
auDA RDAP (.au, auRdap: "polite")	≤ 10
DNS / email_security	10,000 at about 27.5 per second

Why this Actor

	This Actor	Typical WHOIS / DNS Actors
Checks in one run	5 (RDAP, DNS, email, SSL, website)	1–2
Pay only for checks you pick	✅ presets from 1 to 5 checks	❌ flat price
Not charged when a check can't finish	✅ (rate limits, timeouts)	❌
False-positive protection	✅ network failures ≠ site failures	❌ "down" on any timeout
Registry rate limiting	✅ per registry, adaptive backoff, rdapStatus per row	❌ or global only
Built-in health check	✅ selfTest mode	❌
"No DNS = available" mistakes	✅ never: availability only from the registry; DNS delegation can only prove registered	❌ common (users report "available" domains that aren't)
Missing expiry dates explained	✅ a note says which fields the registry withholds	❌ silent null
Bulk input	✅ paste, or link a CSV / TXT / Google Sheet	❌ paste only

Quick start

1. Click Try for free.
2. Paste your domains (one per line), or put a link to a CSV / TXT file or a Google Sheet (shared as "Anyone with the link can view") in domainsFileUrl. A column named domain, website, url or email is used if there's a header row; otherwise the first column. URLs and email addresses work too: https://www.shop.com/cart and jane@shop.com both become shop.com.
3. Pick a preset ("What to check"). The default, Full audit, runs all 5 checks.
4. Click Start. Most runs of a few hundred domains finish in a minute or two.
5. Open the Output tab. Pick a view (Overview, Flags & evidence, Expiry monitor, Email security, Website & SSL, or Flat export). Download as CSV, Excel or JSON.

Via API:

curl -X POST "https://api.apify.com/v2/acts/<username>~domain-intel/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"domains":["github.com","example.org"],"preset":"email_security"}'

Presets: pay only for what you need

Preset	Checks	Typical use
full (default)	rdap, dns, email, ssl, http	Full agency / security audit
expiry_monitor	rdap, ssl	Weekly domain and certificate renewal watch
email_security	email	SPF/DKIM/DMARC deliverability audit
website_health	ssl, http	Uptime-style sanity check of a site list
ownership	rdap, dns	Who is the registrar, and where DNS is hosted
lead_qualification	rdap, email, http	Is this business real, active and on Google/Microsoft?
custom	your choice in Custom checks	Anything else

Each check is billed separately, and only when it produced a conclusive answer. The row field chargedChecks shows exactly what you paid for.

Three examples

1 — Full audit of a healthy domain

Input:

{ "domains": ["github.com"] }

Output (trimmed):

{
  "domain": "github.com", "rdapStatus": "ok", "registrar": "MarkMonitor Inc.",
  "createdAt": "2007-10-09T18:20:50Z", "expiresAt": "2028-10-09T18:20:50Z", "daysUntilExpiry": 709,
  "dnsProvider": "NS1", "emailProvider": "Microsoft 365",
  "spfPolicy": "softfail (~all)", "dmarcPolicy": "quarantine",
  "dkimSelectorsFound": ["google", "k1", "s1", "s2", "selector1", "selector2"], "emailSecurityGrade": "B",
  "sslStatus": "ok", "sslIssuer": "Sectigo Limited", "sslDaysLeft": 88,
  "websiteStatus": "up", "httpStatus": 200,
  "checkStatus": { "rdap": "ok", "dns": "ok", "email": "ok", "ssl": "ok", "http": "up" },
  "flags": [], "chargedChecks": ["rdap", "dns", "email", "ssl", "http"]
}

2 — Website health on a broken certificate

Input:

{ "domains": ["expired.badssl.com"], "preset": "website_health" }

Output (trimmed):

{
  "host": "expired.badssl.com", "sslStatus": "invalid_cert", "sslValidTo": "2015-04-12T23:59:59.000Z",
  "websiteStatus": "tls_error",
  "flagDetails": [
    { "flag": "SSL_EXPIRED", "confidence": "high", "evidence": "Certificate expired 2015-04-12T23:59:59.000Z" }
  ],
  "chargedChecks": ["ssl", "http"]
}

3 — Australian domains: email audit, with .au RDAP left off

Input:

{ "domains": ["abc.net.au"], "preset": "lead_qualification" }

Output (trimmed):

{
  "domain": "abc.net.au", "rdapStatus": "skipped_registry_policy",
  "emailProvider": "Proofpoint", "spfPolicy": "softfail (~all)", "dmarcPolicy": "reject", "emailSecurityGrade": "B",
  "websiteStatus": "up",
  "notes": ["RDAP lookups for rdap.cctld.au are switched off (see the \"auRdap\" input)"],
  "chargedChecks": ["email", "http"]
}

The RDAP part was skipped, so it was not charged.

Output fields

Field	Meaning
rowStatus	checked, invalid_input (couldn't be read as a domain — see reason), or not_processed (spend limit or maxDomains reached — see reason). Only checked rows are ever charged
reason	Why a row is invalid_input / not_processed, e.g. "unknown top-level domain"
input / domain / host / tld	Your input; the registrable domain (used for RDAP); the host checked for DNS/SSL/HTTP (subdomain kept, www. stripped); the public suffix. Internationalised names are converted to their ASCII (punycode) form
domainUnicode	The readable form of an internationalised domain, e.g. münchen.de for xn--mnchen-3ya.de
checksRun	Checks that ran for the row
rdapStatus	See the table below
rdapRetryAfter	If rate-limited: the earliest time the registry said to retry
registered	true / false when the registry answered; true when there's no registry data but the domain is delegated in DNS; null = unknown (never read null as "available")
registeredSource	rdap, dns_delegation, or null
registrar, createdAt, expiresAt, updatedAt	From the registry's RDAP record
domainAgeDays, daysUntilExpiry	Derived from the dates above (negative = expired)
statuses, dnssec	EPP status codes and whether DNSSEC is enabled
nameservers, dnsProvider	Authoritative nameservers and the detected provider (Cloudflare, Route 53, …)
dnsStatus	ok, nxdomain (name does not exist), servfail (broken DNS), partial / inconclusive (some lookups timed out), not_run
a, aaaa, mx, txt, caa	Raw DNS records (when dns is selected)
emailProvider, hasMx	Mailbox or gateway provider detected from MX (Google Workspace, Microsoft 365, Proofpoint, Mimecast, …)
spf, spfPolicy	SPF record and its all policy (hardfail / softfail / neutral / pass-all)
dmarc, dmarcPolicy, dmarcSource	DMARC record, its policy, and where it came from (the host, or the parent domain per RFC 7489)
dkimSelectorsFound	Which common DKIM selectors have a public key
emailSecurityGrade, emailGradeNote	A–F composite of SPF + DMARC + DKIM (null if the lookups were inconclusive or the host has no email setup at all). emailGradeNote explains special cases, e.g. a subdomain that sends no mail but inherits the organisation's sp=reject DMARC policy
sslStatus	ok, invalid_cert, no_https (port 443 refused), inconclusive, not_run
sslIssuer, sslValidFrom, sslValidTo, sslDaysLeft, sslCoversDomain, sslTrusted, sslVerifyError, tlsProtocol	Certificate details
websiteStatus	up, up_bot_protected (a WAF challenged us, but the site is up), http_error, down, tls_error, no_dns, inconclusive, not_run
httpStatus, finalUrl, httpsRedirect, hsts, server, hosting, pageTitle, responseMs	Home page details. httpsRedirect is null when it couldn't be assessed
servedOver	https or http: which probe httpStatus came from (e.g. a tls_error site that still answers over plain http)
httpsResult, httpResult	The raw outcome of the https:// and http:// probes
checkStatus	One status per check: {rdap, dns, email, ssl, http} (CSV export: checkStatus.rdap, checkStatus.dns, … columns)
flags, flagsText	Flag names, as a list and as one CSV-friendly string
flagDetails	[{flag, confidence, evidence}], the reason for each flag
notes	Plain-English caveats (e.g. why something was inconclusive)
errors	Raw error messages, per check
chargedChecks	The checks you were billed for on this row
checkedAt	Timestamp

rdapStatus values

Value	Meaning	Charged?
ok	The registry returned the record	yes
not_registered	The registry says the name doesn't exist (HTTP 404)	yes
rate_limited_retry_later	The registry throttled us beyond your wait budget, or the per-run safety cap was reached. Re-run these rows after rdapRetryAfter	no
unsupported_tld	The TLD publishes no RDAP server (some ccTLDs)	no
skipped_registry_policy	Lookups for this registry are switched off (.au by default)	no
error	Network or registry error after all retries	no
not_run	rdap wasn't selected	no

Flags

A flag is raised only when a check gave a conclusive answer about the target. Timeouts, connection resets, proxy errors and registry throttling never create a flag; they add a note instead.

• high confidence: directly observed.
• medium confidence: a strong signal with a known caveat, explained in evidence.

Flag	Raised when	Confidence
NOT_REGISTERED_POSSIBLY_AVAILABLE	RDAP 404 (high if DNS is also empty; medium if nameservers still exist)	high / medium
DOMAIN_EXPIRED / DOMAIN_EXPIRING_SOON	Registry expiry date is past / within expiryWarningDays	high
DOMAIN_IN_REDEMPTION_OR_PENDING_DELETE	The registry status says so (drop-catch candidates)	high
NEWLY_REGISTERED	Registered less than 90 days ago (fraud / lead-quality signal)	high
PARKED	Nameservers belong to a parking or for-sale service	medium
DOMAIN_NOT_IN_DNS	NXDOMAIN: the name isn't delegated (unregistered, expired, suspended)	high
DNS_BROKEN	Both resolvers return SERVFAIL (lame delegation / broken DNSSEC)	medium
NO_WEBSITE_DNS	DNS answers, but there's no A/AAAA record	high
NO_SPF / NO_DMARC	Missing records (high if the domain receives mail; medium for no-mail domains, which should still publish them to stop spoofing)	high / medium
SPF_MULTIPLE_RECORDS	More than one SPF record (a permanent error for receivers)	high
SPF_ALLOWS_ANYONE	SPF +all	high
DMARC_MONITOR_ONLY	DMARC p=none	high
DMARC_INVALID	DMARC record without a valid p=	high
SSL_EXPIRED / SSL_EXPIRING_SOON	Certificate past / within expiryWarningDays	high
SSL_NAME_MISMATCH	The certificate doesn't cover the host	high
SSL_UNTRUSTED	Self-signed, revoked or untrusted chain (medium for incomplete chains, which some browsers repair)	high / medium
WEBSITE_DOWN	Connection refused on both 443 and 80 (timeouts don't count)	high
HTTP_SERVER_ERROR	5xx on two attempts	high
HTTP_NOT_FOUND	Home page returns 404 / 410	high
HTTP_ACCESS_DENIED	401 / 403 with no bot wall detected (may be intentional)	medium
NO_HTTPS	Site answers on http, but 443 refuses connections	high
NO_HTTPS_REDIRECT	http:// serves content without redirecting to https://	high

Use cases

• Agency audits: run a client or prospect list through full for a one-page health report: expiring domains and certificates, missing DMARC, broken redirects, who hosts what.
• Lead qualification: lead_qualification tells you whether a business is real and active. You get domain age, mailbox provider (Google Workspace / Microsoft 365 = an established business), whether the website is live, and the page title.
• Security & email deliverability: email_security across all your sending domains and brand look-alikes. NO_DMARC, DMARC_MONITOR_ONLY and SPF_ALLOWS_ANYONE are the spoofing risks to fix first.
• Expiring-domain monitoring: schedule expiry_monitor weekly. Filter on DOMAIN_EXPIRING_SOON, SSL_EXPIRING_SOON or DOMAIN_IN_REDEMPTION_OR_PENDING_DELETE, and connect an Apify integration (Slack, email, Zapier, Make) to get alerts.

Registry rate limits (RDAP)

Registries throttle RDAP. This Actor is built to stay on their good side:

• Per-registry queues. Each registry server has its own concurrency cap (rdapConcurrencyPerRegistry, default 2) and its own spacing (rdapMinIntervalMs, default 300 ms). Built-in profiles give .com/.net (Verisign) and .org more headroom and .au much less.
• Adaptive backoff. On HTTP 429 the spacing doubles and requests retry with exponential backoff plus jitter (rdapMaxRetries). Short Retry-After values are honoured. The spacing relaxes again after successes.
• Circuit breaker. If a registry asks for a longer wait than rdapMaxWaitSecs (default 60 s), that row and every later row for the same registry come back immediately as rate_limited_retry_later, with rdapRetryAfter set. They are not charged and don't stall the run.
• Built-in safety profiles. .org (PIR): 1 request at a time, 500 ms apart, at most 250 per run. .au (auDA): off by default. Your settings are always applied, but for these registries they can only make lookups slower or fewer, never faster.
• Fine-tuning. rdapRegistryOverrides lets you set concurrency, minIntervalMs, maxRetries, maxWaitSecs and maxPerRun per registry host. rdapConcurrencyPerRegistry / rdapMinIntervalMs apply to every registry (the stricter of your value and the built-in profile wins).
• One retry time per registry. All rows rate-limited by the same registry share one rdapRetryAfter, so you can re-run them together.

About .au domains

auDA's WHOIS policy restricts automated querying of WHOIS/RDAP and blacklists IPs for 24 hours when limits are exceeded. .au records also don't include creation or expiry dates. So .au RDAP is off by default (auRdap: "off"). DNS, email, SSL and website checks still run normally for .au domains.

auRdap: "polite" allows a handful of slow lookups per run (1 at a time, 10 s apart, at most 10). Please make sure your use complies with the auDA WHOIS Policy.

Daily health check (self-test)

Run the Actor with:

{ "selfTest": true }

This ignores domains and runs 14 assertions against stable public test targets: github.com, example.com, three badssl.com certificate fixtures, a never-registered domain and an invalid entry. If any assertion fails, the run fails. So a daily Apify Schedule plus the built-in "run failed" notification gives you a free health check. The full report is saved as SELF_TEST in the run's key-value store. Self-test rows are not charged.

Every run also saves a RUN_SUMMARY record with:

• counts per rdapStatus, websiteStatus and flag;
• the checks charged;
• per-registry request and rate-limit statistics.

FAQ

Is this legal? Yes. RDAP is the registries' official public interface, DNS is public, and the TLS/HTTP checks make one ordinary request per site. No personal data is collected beyond what registries publish. Most registrant contact details are redacted under GDPR and by registry policy. Respect each registry's terms (see the .au section).

Why is registrar or expiresAt empty for my .au / .de / .io / .info domain? Either the registry publishes no RDAP (unsupported_tld: e.g. .de, .io, .co), the registry was skipped (skipped_registry_policy), or the registry withholds that field. The row's notes say which. A missing expiry date is never treated as expired. For TLDs without RDAP, registered is still filled in from DNS delegation where possible (registeredSource: dns_delegation).

Can I use it as a domain availability checker? Yes, for TLDs with RDAP: registered: false comes straight from the registry. We never call a domain available just because it has no DNS records. Expired, parked and held domains often have none. Premium or reserved names can still be unregistrable, so confirm at a registrar before you buy.

How do I check 1,000+ domains? Put them in a Google Sheet or CSV and pass the link as domainsFileUrl (up to 100,000 rows), or call the API with a domains array. There is no artificial per-plan cap.

I got rate_limited_retry_later. What now? Filter the dataset on that status and re-run just those domains after rdapRetryAfter. You weren't charged for those lookups.

Why does a site I know is up show inconclusive? We couldn't reach it from Apify's network (timeout, connection reset or geo-block). Rather than call it down, we leave it unflagged. Re-run, or check httpsResult / httpResult for details.

What is up_bot_protected? Cloudflare, Akamai, DataDome or a similar firewall challenged our request. The site is up; its content just isn't served to robots. No HTTP flag is raised.

Does a missing DKIM selector mean DKIM is off? Not necessarily. DKIM keys can only be found by name. We probe 12 common selectors, and you can add your own in dkimSelectors.

How fast is it? About 2–3 domains per second with the full preset, about 10 per second for ownership on .com/.net, and about 25 per second for email_security. See How fast is it / how big can I go? above.

Some inputs came back as invalid_input. They couldn't be read as a domain (e.g. a typo'd TLD, an IP address, or a bare public suffix like com.au). reason says which. They're free. Inputs like Name <info@example.com>, quoted values and mailto: links are understood.

Which column of my sheet is used? A column headed domain, website, url or email if there is one; otherwise the column with the most domain-looking values. A header row is skipped automatically. The log names the column it used.

Can I use subdomains? Yes. blog.example.com checks DNS, SSL and HTTP on the subdomain, and RDAP on example.com. DMARC falls back to the parent domain's policy, as mail receivers do.

Input reference

Field	Default	Notes
domains	—	Domains, URLs or emails
domainsFileUrl	—	Link to a CSV / TXT / Google Sheet (public). Combined with domains
maxDomains	0 (no cap)	Check at most this many domains; the rest get free not_processed rows and go into REMAINING
preset	full	See presets
checks	all	Used when preset = custom
expiryWarningDays	30	Threshold for the *_EXPIRING_SOON flags
dkimSelectors	12 common selectors
auRdap	off	off or polite
rdapConcurrencyPerRegistry	2	Built-in profiles (.org, .au) can only be made stricter
rdapMinIntervalMs	300	Adaptive; built-in profiles can only be made slower
rdapMaxRetries	3
rdapMaxWaitSecs	60	Above this, the row is marked rate_limited_retry_later
rdapRegistryOverrides	{}	Per-host tuning
maxConcurrency	10	Domains in parallel
dnsServers / dnsMode	1.1.1.1, 8.8.8.8 / auto	Falls back to DNS-over-HTTPS automatically
selfTest	false	Health-check mode

Support

Found a wrong result? Open an issue on the Actor's Issues tab with the domain and the row's checkStatus, flagDetails and notes. Those three fields usually pinpoint the cause within minutes.