WHOIS RDAP Scraper

🔎 WHOIS RDAP Scraper – Domain Registration, Expiry & Nameserver Lookup

The WHOIS RDAP Scraper performs bulk domain WHOIS lookups over RDAP and returns structured registration data — registrar, creation date, expiry date, last update, EPP status codes, and authoritative nameservers — for every domain you submit. Instead of parsing the free-text WHOIS output that varies by registry, this WHOIS scraper reads the modern RDAP (Registration Data Access Protocol) endpoints and hands back clean, typed JSON that drops straight into a spreadsheet or database.

RDAP is the IETF successor to port-43 WHOIS. It is what registries and registrars are required to serve, it returns JSON rather than unparseable text, and it is consistent across TLDs in a way legacy WHOIS never was. That is the difference between a domain lookup you can automate and one you have to babysit with regex.

Typical users are domain investors monitoring expiry windows, security analysts profiling infrastructure, brand protection teams tracking lookalike registrations, and IT operations teams who simply need a reliable inventory of when their own portfolio renews.

---

📊 What Data Can You Extract with This WHOIS Scraper?

Each record covers one domain and groups the RDAP response into a flat, analysis-ready shape.

Category	Fields	What it gives you
🌐 Domain identity	domain	The domain name the record describes, as returned by the registry
✅ Lookup outcome	success, error	Whether the RDAP query resolved, and the error message when it did not
🏛️ Registrar detail	registrar	Registrar object containing the sponsoring registrar's name and IANA ID
📅 Lifecycle dates	registrationDate, expirationDate, lastUpdated	Creation, expiry, and last-modified timestamps in ISO 8601
🔐 Registry status	status	Array of EPP status codes such as client transfer prohibited or server delete prohibited
🖧 DNS delegation	nameservers	Array of authoritative nameserver hostnames delegated for the domain

The single most valuable field for most workflows is status. EPP status codes are the registry's own statement about what can be done to a domain right now — whether transfers are locked, whether deletion is prohibited, whether the domain sits in a redemption or pending-delete state. A domain carrying clientTransferProhibited is protected against hijack attempts; one showing pendingDelete is about to become available. That single array answers questions that expiry dates alone cannot.

---

🌟 Key Features of the WHOIS Scraper

Feature	Description
📡 RDAP-native lookups	Queries RDAP endpoints rather than parsing legacy port-43 WHOIS text, giving consistent structured output
📦 Bulk domain processing	Submit an array of domains in a single run and get one dataset record per domain
🎯 Single-domain mode	The domain field handles quick one-off checks without building a list
📅 Full lifecycle timestamps	Registration, expiration, and last-updated dates returned in ISO 8601 for direct date arithmetic
🔐 EPP status codes	The status array exposes registry lock states, redemption, and pending-delete signals
🖧 Nameserver extraction	nameservers returns the delegated authoritative servers, useful for hosting and DNS provider mapping
🏛️ Registrar identification	The registrar object carries the sponsoring registrar's name and numeric IANA ID
✅ Per-domain success flag	success plus error make failed lookups explicit and easy to filter or retry
⚡ No browser required	Runs as a lightweight HTTP client, keeping memory use and run duration low

---

🚀 Why Choose This WHOIS Scraper?

RDAP instead of legacy WHOIS parsing. Traditional WHOIS returns free text whose layout differs by registry, so scraping it means maintaining a regex per TLD. This WHOIS scraper reads RDAP JSON, which is structured at the source — the fields you get are the fields the registry publishes, not the fields a parser guessed at.

Built for portfolios, not single lookups. The domains array lets you check an entire portfolio in one run, and every record carries its own success flag, so a single unreachable registry never invalidates the rest of the batch.

Registry status codes are exposed, not discarded. Many domain lookup tools return only dates. Surfacing the full status array gives you the transfer locks, delete prohibitions, and redemption states that matter for security, brand protection, and drop-catching workflows.

Clean, flat output that joins easily. With domain echoed on every record and dates in ISO 8601, results merge into an existing portfolio spreadsheet or SQL table without transformation work.

---

📥 Input

{
  "domain": "example.com",
  "domains": [
    "google.com",
    "github.com",
    "wikipedia.org"
  ]
}

🔧 WHOIS Scraper Input Fields

Field	Type	Required	Default	Description
domain	string	No	example.com	A single domain to check
domains	array	No	["google.com", "github.com", "wikipedia.org"]	A list of domains to check

Neither field is strictly required, and both ship with working defaults so the Actor runs successfully the first time you open it.

💡 Input Examples

Check one domain quickly

{
  "domain": "anthropic.com"
}

Audit a portfolio of domains in a single run

{
  "domains": [
    "example.com",
    "example.net",
    "example.org",
    "example.co.uk"
  ]
}

Combine a primary domain with a watchlist

{
  "domain": "mybrand.com",
  "domains": [
    "mybrand.net",
    "my-brand.com",
    "mybrandshop.com"
  ]
}

---

📤 Output

{
  "domain": "GOOGLE.COM",
  "success": true,
  "status": [
    "client delete prohibited",
    "client transfer prohibited",
    "client update prohibited",
    "server delete prohibited"
  ],
  "nameservers": [
    "NS1.GOOGLE.COM",
    "NS2.GOOGLE.COM",
    "NS3.GOOGLE.COM",
    "NS4.GOOGLE.COM"
  ],
  "registrar": {
    "name": "MarkMonitor Inc.",
    "id": "292"
  },
  "registrationDate": "1997-09-15T04:00:00Z",
  "expirationDate": "2028-09-14T04:00:00Z",
  "lastUpdated": "2019-09-09T15:39:04Z"
}

🧾 WHOIS Scraper Output Fields

Field	Type	Description
domain	string | null	Domain name of the record
success	boolean | null	Whether the RDAP lookup succeeded
status	array | null	Registry status codes reported for the domain
nameservers	array | null	Authoritative nameserver hostnames for the domain
registrar	object | null	Registrar details, including name and id
registrationDate	string | null	Date the domain was first registered
expirationDate	string | null	Date the current registration expires
lastUpdated	string | null	Date the registry record was last modified
error	string | null	Error message, if the lookup failed

Registries commonly return domain names and nameserver hostnames in uppercase, as shown in the sample. Lowercase them during post-processing if you are joining against data from another source.

---

💻 How to Use the WHOIS Scraper (Step by Step)

Step 1: Open the WHOIS RDAP Scraper Actor

Sign in to Apify and open the Actor. The input form shows two fields: Single Domain and List of domains. Both are prefilled, which means you can press Start immediately and see a real RDAP response before configuring anything of your own.

Step 2: Decide between single-domain and bulk domain lookup

Use the domain field for quick, ad-hoc checks — verifying an expiry date before a renewal, or confirming which registrar sponsors a domain you are about to acquire. Use the domains array when you are auditing a portfolio, monitoring a brand watchlist, or building an inventory. The two can be combined in the same run.

Step 3: Enter your domains

Type domains as bare hostnames — example.com, not https://example.com/path. Include the full registrable domain including its TLD. For country-code and multi-label suffixes such as .co.uk, supply the whole registrable name rather than a subdomain, since RDAP records exist at the registrable-domain level.

Step 4: Run the WHOIS scraper and monitor progress

Press Start. The log reports each domain as it is queried. Because the Actor issues plain HTTP requests to RDAP endpoints rather than driving a browser, batches of domains complete quickly and memory usage stays modest.

Step 5: Review the WHOIS data in the dataset

Open the Dataset tab. Each row is one domain with its registration lifecycle, registrar, status codes, and nameservers. Scan the success column first — any false rows carry an error message describing why the RDAP lookup did not resolve, typically an unsupported TLD or an unreachable registry service.

Step 6: Export and analyse the domain registration data

Export to CSV, JSON, Excel, XML, or HTML, or read the dataset through the Apify API. For expiry monitoring, sort by expirationDate and compute days remaining. For infrastructure analysis, group by the registrar name or by the second-label of each nameserver hostname to see which providers dominate your portfolio.

Step 7: Schedule recurring WHOIS monitoring

Domain records change: renewals push out expirationDate, transfers change registrar, DNS migrations rewrite nameservers, and lock changes alter status. Schedule the WHOIS scraper to run on a regular cadence and diff each run against the previous one to catch unauthorised transfers, unexpected DNS changes, and approaching expiries while there is still time to act.

---

🔌 API Access & Integrations

Run a bulk WHOIS lookup synchronously and get the dataset items in the response:

curl -X POST "https://api.apify.com/v2/acts/scrapers-hub~whois-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "domains": ["google.com", "github.com", "wikipedia.org"]
  }'

Check a portfolio for upcoming expiries with the Python client:

from datetime import datetime, timezone
from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run = client.actor("scrapers-hub/whois-scraper").call(run_input={
    "domains": ["example.com", "example.net", "example.org"]
})

now = datetime.now(timezone.utc)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if not item.get("success"):
        print("FAILED", item.get("domain"), item.get("error"))
        continue
    expiry = item.get("expirationDate")
    if expiry:
        days = (datetime.fromisoformat(expiry.replace("Z", "+00:00")) - now).days
        print(item["domain"], "expires in", days, "days via",
              (item.get("registrar") or {}).get("name"))

The WHOIS scraper also connects to Zapier, Make, Google Sheets, and Slack, and can fire webhooks when a run finishes — handy for posting expiry alerts into a team channel automatically.

---

💡 Best Use Cases for WHOIS and RDAP Domain Data

📅 Domain expiry monitoring and renewal alerts

Pull expirationDate for every domain you own and calculate the days remaining. Feed the results into a scheduled job that posts a Slack alert whenever a domain crosses a warning threshold. An accidentally lapsed domain is one of the most expensive avoidable outages a business can have, and this is the cheapest possible insurance against it.

🛡️ Brand protection and typosquat surveillance

Maintain a watchlist of lookalike domains in the domains array. A previously unregistered variant that suddenly returns success: true with a fresh registrationDate is a strong early signal of a phishing or impersonation campaign, and the registrar and nameservers values give you the parties to contact for takedown.

🔐 Security research and infrastructure profiling

When investigating a suspicious domain, registrationDate tells you how new it is — newly registered domains are disproportionately associated with abuse — while nameservers reveals the hosting or DNS provider behind it. Combining those with the status array builds a quick reputational profile without any paid threat intelligence feed.

💰 Domain investing and drop-catching research

Domain investors care about the transitions encoded in status. Codes indicating redemption or pending-delete states, cross-referenced against expirationDate, identify names about to return to the available pool. Bulk-checking candidate lists in one run turns a manual per-domain chore into a data pipeline.

🏢 IT asset inventory and portfolio governance

Large organisations accumulate domains across departments, agencies, and acquisitions. Running the WHOIS scraper over the whole list produces an authoritative inventory keyed on domain, showing which registrar sponsors each name and whether registry locks are in place — the foundation of any domain governance policy.

🔄 Transfer and DNS change detection

Diff registrar and nameservers between scheduled runs. An unexpected registrar change may indicate an unauthorised transfer; an unexpected nameserver change may indicate a compromised DNS account or an undocumented migration. lastUpdated corroborates when the registry record actually changed.

🤝 Due diligence on acquisitions and partnerships

Before acquiring a domain, a brand, or a company, confirm what the registry actually says: how long the domain has existed via registrationDate, how long the current registration runs via expirationDate, who sponsors it via registrar, and whether transfer locks in status will complicate the handover.

---

⚙️ Tips for Better WHOIS Scraping Results

• Submit registrable domains only. RDAP records exist for the registrable name, so use example.co.uk rather than shop.example.co.uk, and strip protocols, paths, and www. prefixes before submitting.
• Normalise case downstream. Registries frequently return uppercase domains and nameservers. Lowercase both before joining against data from other systems, or you will create phantom mismatches.
• Always check success before trusting nulls. A null expirationDate on a failed lookup means "not retrieved", not "no expiry". Filter on success first, then read the error message on the remainder.
• Batch domains rather than looping single runs. The domains array is the efficient path for portfolios — one run, one dataset, far less orchestration overhead than one run per domain.
• Expect coverage gaps on some TLDs. RDAP adoption is near-universal for gTLDs but uneven across some country-code registries. Domains on those TLDs may return an error rather than a full record.
• Store historical snapshots. The real value in WHOIS monitoring comes from change detection. Keep each run's dataset and compare registrar, nameservers, and status over time rather than only reading the latest values.

---

🛠️ Troubleshooting

A domain returned success: false with an error — why? The most common causes are a TLD whose registry does not publish an RDAP service, a temporarily unreachable registry endpoint, or a domain that is simply not registered. Read the error message for the specific reason, and retry transient failures on a later run.

Some fields come back null even though the lookup succeeded. RDAP responses vary by registry. Some registries publish a complete lifecycle record; others omit lastUpdated or return a sparse registrar object. Null means the registry did not include that element, not that the WHOIS scraper failed to read it.

Registrant name, email, and address are missing. Contact-level personal data is redacted in public RDAP responses under GDPR and ICANN's temporary specification. The output fields available are the registry-level ones documented above; personal registrant contact details are not part of them.

My subdomain lookup returned nothing. Registration data exists at the registrable-domain level. Query example.com rather than api.example.com, then apply the result to all subdomains beneath it.

A large batch of domains ran slowly or partially failed. Registry RDAP services apply their own rate limits. Split very large portfolios into several runs, and re-run any domains whose error values suggest throttling rather than a genuine absence of data.

---

❓ Frequently Asked Questions About WHOIS Scraping

What is the difference between WHOIS and RDAP? WHOIS is the legacy port-43 protocol returning free-text output that differs by registry. RDAP is its JSON-based successor with a consistent, machine-readable structure. This WHOIS scraper queries RDAP, which is why the output is reliably typed.

How many domains can I check in one run? Provide as many as you need in the domains array. Very large portfolios are best split across several runs so that registry-side rate limits do not affect part of the batch.

Do I need to supply both domain and domains? No. Both are optional and both have defaults. Use domain for a single quick check, domains for bulk lookups, or both together.

Can this WHOIS scraper return the registrant's name or email? No. Public RDAP responses redact personal contact data for privacy compliance. The Actor returns registrar, dates, status codes, and nameservers.

What do the values in the status array mean? They are registry status codes describing what operations are currently permitted on the domain — for example transfer, update, or delete prohibitions applied by the client or the server. They are the registry's authoritative statement about the domain's current state.

What format are the dates in? ISO 8601 with a UTC offset, as in 2028-09-14T04:00:00Z, so they parse directly in Python, JavaScript, SQL, and spreadsheet software.

Does this Actor work with country-code TLDs? It works with any TLD whose registry publishes an RDAP service. Coverage is near-universal for generic TLDs and varies for some country-code registries, which may return an error instead.

Can I use the WHOIS scraper for domain expiry monitoring? Yes — that is one of its main uses. Schedule regular runs, read expirationDate, and alert when the remaining days drop below your threshold.

How do I detect an unauthorised domain transfer? Compare the registrar object and the status array between consecutive runs. A registrar change you did not initiate, or the disappearance of a transfer-prohibited status code, warrants immediate investigation.

Why are domain names and nameservers returned in uppercase? That is how many registries publish them. Domain names are case-insensitive, so lowercase them in post-processing if your downstream systems expect it.

Does the Actor use a proxy or a browser? No. It makes direct HTTP requests to RDAP endpoints without a headless browser, which keeps runs fast and lightweight.

Can I integrate the WHOIS scraper with Slack or Google Sheets? Yes. It works with Zapier, Make, Google Sheets, and Slack integrations, and supports webhooks that fire on run completion.

What export formats are available? Apify datasets export as JSON, CSV, Excel, XML, and HTML, and can also be read directly through the dataset API.

Is scraping WHOIS and RDAP data legal? RDAP data is published by registries specifically for public access, and the personal elements are redacted at source. You remain responsible for how you use the results, including compliance with the applicable data protection and anti-abuse rules in your jurisdiction.

Can I get a customised version of this WHOIS scraper? Yes. Email scraperhubapi@gmail.com to discuss custom domain data extraction, additional fields, or integration work.

---

🆘 Support & Feedback

Found a domain that should resolve but does not, or a field that looks wrong? Report it on the Issues tab of the Actor page. Include the domain and the run ID so the RDAP response can be reproduced quickly.

For custom work — larger domain monitoring pipelines, change-detection alerting, or a tailored version of this WHOIS scraper — email scraperhubapi@gmail.com.

If the WHOIS scraper is useful to you, please leave a review on the Apify Store. Feedback and ratings directly influence which features get built next.

---

⚖️ Disclaimer

The WHOIS RDAP Scraper retrieves publicly published domain registration data from RDAP services operated by domain registries and registrars. It does not bypass authentication, access private registrar systems, or attempt to recover redacted information.

You are responsible for how you use the domain data this WHOIS scraper produces. Registrant contact details are redacted at source under privacy regulations including the GDPR; where any data you handle alongside these results identifies an individual, you act as the data controller and must establish a lawful basis, respect data subject rights, and apply appropriate retention limits. Follow the terms of service of the registries and services involved in your workflow, and observe reasonable request rates.

Registry data is provided as-is and may be incomplete, delayed, or unavailable for certain TLDs. Verify critical facts with the registrar of record before relying on them for legal, financial, or compliance decisions.

If you believe data returned by this Actor should be removed or corrected, email scraperhubapi@gmail.com with the details and the request will be reviewed.