RDAP Domain Scraper: Modern WHOIS Replacement

RDAP Domain Scraper looks up domain registration data through RDAP, the structured, JSON-based successor to WHOIS, and returns registrar, creation and expiry dates, status codes, nameservers, DNSSEC state and abuse contacts, one record per domain.

RDAP is what ICANN mandated to replace WHOIS's unparseable free text. Every domain is routed to its authoritative registry using IANA's published bootstrap registry, and redirects to registrar RDAP servers are followed automatically.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/whois-rdap-scraper.md

What you get

Output field	Meaning
domain	Normalised registrable domain
registrar	Sponsoring registrar name
registrarIanaId	IANA-assigned registrar number, where published
createdDate, expiryDate, updatedDate	Registration, expiration and last-changed events
status	EPP status codes (client transfer prohibited, server delete prohibited, …)
nameservers	Delegated nameserver hostnames, lower-cased, root dot stripped
dnssec	Whether the delegation is signed
registrantCountry, registrantOrg	Registrant details where the registry publishes them
abuseEmail	Registrar's abuse contact address
rdapServer	The registry server that actually answered
rawResponse	The complete unparsed RDAP object (opt-in)
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the options used, every domain that failed, and every domain the cap left out.

Common use cases

• Portfolio expiry monitoring. Sweep your domains on a schedule and alert before renewal dates.
• Due diligence. Check registration age, registrar and status codes during an acquisition.
• Brand protection. Detect newly registered lookalike domains and find their abuse contacts.
• Threat intelligence. Enrich indicators with registration age, nameservers and DNSSEC state.

Quick start

A handful of domains:

{
 "domains": ["apify.com", "wikipedia.org", "bbc.co.uk"]
}

Lean output for a large portfolio sweep, with the run bounded to 500 records:

{
 "domains": ["example1.com", "example2.com", "example3.net"],
 "maxDomains": 500,
 "includeNameservers": false
}

Everything the registry published, unparsed:

{
 "domains": ["lemonde.fr"],
 "includeRawResponse": true
}

Input

Field	Type	Default	Notes
domains	array	-	Required. Bare domains, hostnames with www, or full URLs, all normalised to the registrable name and de-duplicated.
maxDomains	integer	100	Ceiling on lookups for the whole run, and therefore on records returned and on what the run costs. Domains past it are skipped and listed in RUN_SUMMARY.domainsSkipped. 0 means look up every domain in the list.
includeRawResponse	boolean	false	Attach the registry's complete RDAP JSON as rawResponse. Much larger records.
includeNameservers	boolean	true	Include delegated nameserver hostnames.

Limits and cost control

• maxDomains is a total for the run, not a per-domain allowance. The run looks up at most that many domains, one record each, so the number of charged rows can never exceed it. Domains past the cap are never requested and are named in RUN_SUMMARY.domainsSkipped.
• A domain that fails uses one of those slots. The cap counts lookups, so a run can return fewer records than the cap if some of the domains it reached were unregistered or unroutable. Both the failures and the untouched remainder are named in RUN_SUMMARY.
• The default is 100. Pass a bigger number for a bigger portfolio, or 0 to look up everything you listed.
• Out-of-range values are rejected, not reinterpreted. A negative, fractional or non-numeric maxDomains fails the run before any lookup happens, rather than quietly becoming "no limit".
• Only delivered records are charged. Duplicate spellings of one name collapse to a single lookup, and failed or skipped domains cost nothing.

Output example

{
 "domain": "apify.com",
 "registrar": "Amazon Registrar, Inc.",
 "registrarIanaId": "468",
 "createdDate": "2009-06-02T17:14:10Z",
 "expiryDate": "2035-06-02T17:14:10Z",
 "updatedDate": "2026-05-16T16:53:04Z",
 "status": ["client transfer prohibited"],
 "nameservers": [
 "ns-1225.awsdns-25.org",
 "ns-1928.awsdns-49.co.uk",
 "ns-449.awsdns-56.com",
 "ns-839.awsdns-40.net"
 ],
 "dnssec": true,
 "registrantCountry": null,
 "registrantOrg": null,
 "abuseEmail": "trustandsafety@support.aws.com",
 "rdapServer": "https://rdap.verisign.com",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~whois-rdap-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{ "domains": ["apify.com", "bbc.co.uk"] }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/whois-rdap-scraper').call({
 domains: ['apify.com', 'wikipedia.org', 'bbc.co.uk'],
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const d of items) console.log(`${d.domain}, ${d.registrar}, expires ${d.expiryDate}`);

Notes

• Routing comes from IANA, not guesswork. IANA's published bootstrap registry is loaded once per run and maps roughly 1,200 TLDs to their registry RDAP base URL. rdapServer records which server actually answered.
• Bootstrap gaps are filled from verified probes. Several popular TLDs, including .io, .de, .me, .sh, .ac and .cc, are absent from IANA's file, and the public fallback router is generated from that same file so it fails on exactly the same set. This Actor carries a small table of directly verified registry endpoints for them. TLDs with no RDAP service fail with no RDAP service published for .<tld>, which is the accurate answer.
• Contact data is largely GDPR-redacted, and that is not a bug. For most gTLDs, registries publish the registrar and the abuse contact but redact registrant name, organisation and address. registrantOrg and registrantCountry are frequently null for .com, and frequently populated for ccTLDs such as .fr and .io. Nothing is synthesised to fill the gap, do not build a product that promises registrant details.
• jCard parsing is explicit. RDAP contacts arrive as jCard (RFC 7095), a positional JSON encoding of vCard: each property is a 4-element array [name, params, type, value], structured properties such as adr put a 7-slot array in the value position, and there is no key lookup at all. That awkwardness is isolated in one commented helper rather than smeared across the extraction code.
• Bad domains don't kill the run. Unregistered domains, unroutable TLDs, registry outages and entries that are not domains at all are recorded in RUN_SUMMARY.failures by name; the Actor only errors out if every domain fails, and it writes the summary before it does.
• Transient errors are retried. 429, 5xx and network faults get three attempts with exponential backoff and a 30-second timeout.
• Public data only. No authentication, no access-control bypass, no scraping of redacted fields.

FAQ

Is this WHOIS? No, it is RDAP, the ICANN-mandated replacement. RDAP returns structured JSON with consistent field names, so there is no free-text parsing and no per-registrar format guessing.

Why is the registrant blank? GDPR. Most gTLD registries redact registrant contact data from public RDAP. The registrar and abuse contact are still published, and ccTLDs vary, .fr publishes organisation and country, .com does not.

Which TLDs are supported? Roughly 1,200 from IANA's bootstrap file, plus verified endpoints for popular TLDs missing from it. A few ccTLDs still run WHOIS only; those are reported as unsupported rather than silently returning nothing.

Why do some records have a null registrar? A handful of registries (DENIC for .de, for example) publish the domain object without registrar entity data. The fields they do publish, status, nameservers, last changed, are still returned.

What does dnssec: null mean? The registry did not include a secureDNS block. true and false are real answers; null means "not stated".

How many domains can I pass at once? As many as you like, but a run only looks up maxDomains of them — 100 by default. Raise it for a bigger sweep, or set 0 for no ceiling. Domains are looked up sequentially at one request each, so hundreds per run is normal. Be considerate of small ccTLD registries.

I passed 400 domains and got 100 records. Why? The default maxDomains is 100, and it exists so a long list cannot produce a bill you did not intend. The 300 that were not looked up are listed in RUN_SUMMARY.domainsSkipped. Set maxDomains to 400, or to 0, and re-run.

Can I schedule it? Yes, it is designed for scheduled runs. Diff on expiryDate for renewal alerts, or on status and nameservers for hijack detection.

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.