Bulk Domain DNS & RDAP Lookup – SPF, DMARC, MX, Registrar, Expiry

Enrich a list of domains, up to 10,000 per run, with live DNS and registry data:

• DNS through public DNS-over-HTTPS (Cloudflare 1.1.1.1 or Google Public DNS, each falling back to the other): A, AAAA, MX (including RFC 7505 null MX), NS and TXT, with the SPF record, the DMARC record and its policy, and the resolver's DNSSEC validation flag.
• RDAP, the modern, structured replacement for WHOIS, using the official IANA RDAP bootstrap to find each TLD's registry server: registrar and IANA ID, created, expiry and last-updated dates, days to expiry, domain age, EPP status codes, delegated nameservers and DNSSEC (delegationSigned and DS count).

Privacy by design: all registrant, administrative, technical, billing and abuse contact data (names, emails, phones, postal addresses and organisations of contacts) is dropped. Only the registrar's organisation name and IANA ID are read from RDAP entities, through an allow-list. No port-43 WHOIS is ever used.

How it works

1. Normalises your input. URLs, www. and email addresses are reduced to the domain, IDNs are converted to punycode, and duplicates and invalid entries are skipped and reported in OUTPUT.
2. Downloads the IANA bootstrap (data.iana.org/rdap/dns.json) once per run.
3. For each domain it runs the DoH queries in parallel and the RDAP lookup at the TLD's registry. For a subdomain it looks up the parent registration.
4. Polite rate limiting: requests to each RDAP server are spaced (250 ms by default, configurable). On HTTP 429 the actor honours Retry-After (or backs off exponentially), doubles the interval for that server only and retries. Other registries keep going at full speed.
5. Writes one record per domain, in input order, to the dataset.

Input example

{ "domains": ["example.com", "https://www.apify.com/store", "info@bbc.co.uk"], "recordTypes": ["A", "AAAA", "MX", "NS", "TXT"], "includeDmarc": true, "includeRdap": true, "dohProvider": "cloudflare" }

Output example (abridged)

{
  "domain": "apify.com", "registrable": "apify.com", "resolves": true, "nxdomain": false,
  "a": ["3.170.42.100"], "aaaa": ["2600:9000:2870:1a00:9:a03e:6540:93a1"],
  "mx": [{ "priority": 1, "host": "aspmx.l.google.com" }], "nullMx": false, "ns": ["ns-449.awsdns-56.com"],
  "spf": "v=spf1 a mx include:_spf.google.com ... -all", "hasSpf": true, "dmarc": "v=DMARC1; p=reject; ...", "dmarcPolicy": "reject",
  "registered": true, "registrar": "Amazon Registrar, Inc.", "registrarIanaId": "468",
  "createdAt": "2009-06-02T17:14:10Z", "expiresAt": "2035-06-02T17:14:10Z", "updatedAt": "2026-05-16T16:53:04Z",
  "daysToExpiry": 3153, "domainAgeDays": 6342, "status": ["client transfer prohibited"], "dnssec": true,
  "rdapServer": "rdap.verisign.com", "errors": [], "checkedAt": "2026-10-13T17:52:07.422Z"
}

Pricing

Pay per event: $0.0024 per domain (domain-enriched) plus the standard Apify actor-start fee. 10,000 domains cost $24. The actor stops starting new domains when your max cost per run would be exceeded, and every domain you pay for is delivered.

Integrations (Zapier / Make)

Use Apify's Zapier or Make integration, or a webhook on "run succeeded", to enrich new CRM leads with their email domain's MX, SPF and DMARC, to alert on domains with daysToExpiry < 30, or to push results to Google Sheets. You can also call the actor synchronously from the API (run-sync-get-dataset-items) for small batches.

FAQ

Why RDAP instead of WHOIS? RDAP is the IETF standard (RFC 9082/9083) that ICANN requires of gTLD registries. It returns structured JSON over HTTPS and makes rate limits explicit. This actor never uses port-43 WHOIS.

Why are some domains missing registrar and dates? Some ccTLDs, for example .de, .so and .us, are not in the IANA RDAP bootstrap. Those rows still contain the full DNS data, registered: null and an explanatory entry in errors.

Do you return registrant names or emails? No. All contact entities are discarded, including the registrar's abuse contact. Only the registrar's organisation name and IANA ID are kept. Personal-looking contact emails in DNS TXT, SPF and DMARC records are masked by default (rua=mailto:jane.doe@x.com becomes j***@x.com). Role mailboxes such as dmarc@… and DMARC-service addresses are kept, because they are not personal. Set showRawEmails: true to opt out and get the records exactly as published.

How fast is it? About 5 domains per second with the defaults. The limit is the per-registry spacing: Verisign (.com/.net) at 250 ms means about 4 lookups per second. 10,000 .com domains take about 45 minutes. Domains on mixed TLDs run faster.

What does registered: false mean? The registry's RDAP server returned 404. The domain is probably available, but check with a registrar before buying.

dnssecValidated vs dnssec? dnssec is the registry's delegationSigned flag from RDAP. dnssecValidated is the resolver's AD bit for the answers.

What permissions does it need? It runs with limited permissions and only writes to its own default dataset and key-value store.

Licence & legal

DNS answers are public. RDAP responses come from the registries' public RDAP services and are used for lawful domain research only. Personal data is not collected. Respect each registry's terms of use. Email addresses inside TXT/SPF/DMARC records are masked by default (j***@domain); role mailboxes such as dmarc@, postmaster@ or abuse@ and DMARC report services stay readable, and showRawEmails: true disables masking.

Sources and terms:

• IANA RDAP bootstrap registry: https://data.iana.org/rdap/dns.json (overview: https://www.iana.org/assignments/rdap-dns/rdap-dns.xhtml)
• Cloudflare 1.1.1.1 DNS-over-HTTPS: https://developers.cloudflare.com/1.1.1.1/privacy/public-dns-resolver/ and https://www.cloudflare.com/website-terms/
• Google Public DNS: https://developers.google.com/speed/public-dns/terms and https://developers.google.com/speed/public-dns/privacy The actor identifies itself with the User-Agent domain-dns-rdap/0.1 (Apify actor; +https://apify.com).

Running locally

npm ci
npm test            # unit tests (mocked DoH/RDAP)
npm run test:live   # live: example.com, apify.com, bbc.co.uk, NXDOMAIN, Google DoH
npm start           # uses storage/key_value_stores/default/INPUT.json