Bulk DNS & Domain Checker — Records, WHOIS & Expiry

Bulk DNS & Domain Checker — DNS Records, Mail Provider, WHOIS & Expiry

Look up the full DNS and domain profile of thousands of domains at once. This bulk DNS checker takes a list of domains (or URLs, or email addresses) and returns, for each one, its A/AAAA, MX, NS and TXT records, SPF/DMARC/DKIM status, detected mail provider, registrar, creation / last-changed / expiry dates, days-until-expiry, EPP status codes and DNSSEC — no login, no API keys, no coding. If you need to scrape DNS records and extract deep domain intelligence in bulk, this DNS checker is the fastest, cheapest way to do it.

Export every result to JSON, CSV, Excel, or API.

Built for IT and sysadmins, email-deliverability specialists, security and recon analysts, domain investors managing large portfolios, and growth / sales teams enriching company domain lists.

---

⚡ At a glance

Feature	Specification
Lookup Engine	Parallel Node.js DNS resolution + Authoritative RDAP Registry lookup
Supported Records	A, AAAA, MX (with priority), NS, TXT, SPF, DMARC, DKIM hints
Domain Intelligence	Mail Provider, Registrar, Registration Date, Expiry Date, Days to Expiry, EPP Status, DNSSEC
Input Flexibility	Bare domains (stripe.com), Full URLs (https://stripe.com/pricing), Emails (lead@stripe.com)
Throughput / Speed	~100–300 domains/minute (DNS-only up to 1,000+ domains/min)
Authentication / Keys	None required — completely keyless public registry lookups
Proxy Requirement	Not required (Direct connections, zero proxy costs by default)
Export Formats	JSON, CSV, XML, Excel (XLSX), Apify API, Webhooks
Pagination & Batching	1,000+ domain batch processing unlocked with configurable concurrency pool

---

🔎 Why scrape DNS & domain data?

DNS and the public domain registry (WHOIS/RDAP) hold the answers to a surprising number of mission-critical questions: does this domain resolve, where does its mail actually go, is SPF and DMARC configured properly, who is the official registrar, is the domain locked against unauthorized transfer, and when does it expire? For a single domain you can run manual dig or paste it into a web WHOIS tool. For a whole portfolio, a prospect list, or a security audit of hundreds or thousands of domains, doing it one at a time is slow, error-prone, and impossible to keep up to date.

This DNS scraper does it in bulk over native DNS protocols plus RDAP (the modern, structured replacement for legacy WHOIS text), so it is fast, inexpensive, and keyless at scale. You paste a list, it queries each domain in parallel, and you get one clean, flat row per domain that drops straight into a spreadsheet, a BI tool, or your CRM data pipeline.

Because every field is a public record, there is nothing to log into and no rate-limited vendor API to pay for — just the DNS and domain data you need, structured and ready to filter. The checker queries each TLD's authoritative RDAP server directly (resolved live from the IANA bootstrap registry), so registrar and expiry data come back reliably even on big, parallel runs.

Common reasons teams scrape DNS & domain data:

• Domain portfolio monitoring — track expiryDate and daysUntilExpiry across every domain you own so you never miss a renewal and never lose a business-critical name.
• Email-deliverability audits — check mxRecords, hasSpf, hasDmarc and mailProvider across a list of domains to find authentication gaps that send outbound emails to spam.
• Security & infrastructure recon — map nameservers, mail providers, DNSSEC and registrar-lock statusCodes across a target set of domains.
• Lead & account enrichment — add mailProvider and registrar signals to a list of company domains for B2B segmentation and tech-stack targeting.
• M&A and domain-acquisition due diligence — inspect registration age, last-changed date and transfer-lock status before purchasing digital assets.
• Brand protection — bulk-check lookalike and typosquat domains to see which resolve, where their mail points, and who registered them.
• Migration & cutover QA — verify that A, MX and NS records point where they should after a massive DNS migration across many domains at once.

---

✨ What this DNS scraper does

• ✅ Bulk input — paste a few or thousands of domains at once (one per line). URLs like https://www.site.com/page and emails like user@site.com are accepted too; the bare domain is extracted automatically and de-duplicated.
• ✅ Full DNS resolution — IPv4 (aRecords), IPv6 (aaaaRecords), mail exchangers in priority order (mxRecords), nameservers (nsRecords) and TXT records (txtRecords).
• ✅ Email-authentication detection — flags whether each domain publishes SPF (hasSpf), DMARC (hasDmarc, checked on _dmarc.<domain>) and DKIM-related TXT (hasDkim).
• ✅ Mail-provider fingerprinting — resolves the MX host to a recognizable provider (mailProvider): Google Workspace, Microsoft 365, Zoho, Proton, Amazon SES, Mailgun, SendGrid, Proofpoint, Mimecast, Symantec, Yandex, Mail.ru, Tencent and GoDaddy.
• ✅ WHOIS / RDAP enrichment (optional, on by default) — registrar, creation date, last-changed date, expiry date, days-until-expiry, EPP status codes (e.g. clientTransferProhibited) and DNSSEC delegation status.
• ✅ Authoritative-first RDAP — each TLD is mapped to its official RDAP server via the live IANA bootstrap registry and queried directly, with a public bootstrap proxy as fallback. This recovers registrar/expiry data that free proxy endpoints drop under concurrent load, so coverage stays high even on large runs.
• ✅ Parallel & fast — a configurable concurrency pool checks many domains at the same time; DNS and RDAP run in separate pools so RDAP throttling never slows down DNS lookups.
• ✅ Robust error handling — per-lookup timeouts, automatic retries on RDAP 429/5xx, source fallback, and a graceful error field for domains that do not resolve, so one bad domain never breaks the run.
• ✅ Export anywhere — results download as JSON, CSV, or Excel, or stream via the Apify API into Make, Zapier, Google Sheets or a webhook.

---

🎯 Use cases & B2B recipes

1. Automated Domain Portfolio Renewal Monitoring

Keep a consolidated watchlist of corporate domains, client domains, and brand assets. Set up an Apify schedule to run weekly. Filter results where daysUntilExpiry < 45 to alert your infrastructure team before domain names expire or drop into grace periods.

2. B2B Sales Prospect Tech-Stack Enrichment

Sales teams targeting specific IT ecosystems can enrich lead lists. Filter accounts by mailProvider to distinguish Google Workspace organizations from Microsoft 365 enterprises, adjusting cold outreach messaging to match their software environment.

3. Email Deliverability & Anti-Spoofing Audit

Deliverability consultants auditing prospect domains can instantly detect missing SPF (v=spf1) or missing DMARC (v=dmarc1) records. Flagging hasSpf: false or hasDmarc: false provides an immediate, actionable gap report for prospective clients.

4. Brand Protection & Phishing / Typosquatting Recon

Security analysts investigating brand spoofing can supply hundreds of generated permutations of a corporate brand name. Identify which domains resolve (resolves: "true"), what nameservers they use, and whether active mail exchangers (mxRecords) are staged for email spoofing campaigns.

5. Infrastructure & M&A Technical Due Diligence

During technical due diligence for acquisitions, evaluate domain health, registrar pedigree, and transfer security by checking statusCodes for clientTransferProhibited. Verify whether modern DNSSEC delegation is properly deployed.

---

📥 Input parameters

Provide a list of domains and choose whether to include WHOIS. All other settings are optional.

Field	Type	Default	Description
domains	array	["github.com", ...]	Domains to check, one per line (e.g. example.com). URLs and email addresses are accepted too — the domain is extracted and de-duplicated automatically. Leave empty to run a quick demo on a few popular domains.
whoisLookup	boolean	true	Also look up registrar, creation / last-changed / expiry dates, status codes and DNSSEC via RDAP. Turn off for a faster, DNS-only run.
maxItems	integer	0	Safety cap on how many domains are checked in one run. 0 means no limit (process all provided domains).
maxConcurrency	integer	50	How many domains to check in parallel (1–200). Higher is faster; 50 is a good balance.
proxyConfiguration	object	{ useApifyProxy: false }	Apify Proxy settings. Off by default — this check needs no proxy.

DNS-only (fastest for 10,000+ domains)

{
  "domains": ["example.com", "github.com", "cloudflare.com"],
  "whoisLookup": false,
  "maxConcurrency": 100
}

Full check with WHOIS/RDAP

{
  "domains": ["github.com", "mysite.com", "user@client.com", "https://www.shop.io/pricing"],
  "whoisLookup": true,
  "maxConcurrency": 50,
  "maxItems": 0
}

---

📤 Output dataset

You get one flat record per domain — ideal for spreadsheets and filtering. Here is a real item from a fresh run (github.com):

{
  "domain": "github.com",
  "resolves": "true",
  "aRecords": "140.82.114.3",
  "aaaaRecords": "",
  "mxRecords": "github-com.mail.protection.outlook.com",
  "mailProvider": "Microsoft 365",
  "nsRecords": "ns-421.awsdns-52.com, ns-520.awsdns-01.net, dns1.p08.nsone.net, dns2.p08.nsone.net, ns-1283.awsdns-32.org, ns-1707.awsdns-21.co.uk",
  "hasSpf": "true",
  "hasDmarc": "true",
  "hasDkim": "false",
  "txtRecords": "MS=ms44452932 | apple-domain-verification=RyQhdzTl6Z6x8ZP4 | anthropic-domain-verification-4az7qn=if8YWuRRqwLycGJDooumzHtxm | atlassian-domain-verification=jjgw98AKv2aeoYFxiL/VFaoyPkn3undEssTRuMg6C",
  "registrar": "MarkMonitor Inc.",
  "createdDate": "2007-10-09",
  "updatedDate": "2026-09-07",
  "expiryDate": "2028-10-09",
  "daysUntilExpiry": "708",
  "statusCodes": "client delete prohibited, client transfer prohibited, client update prohibited",
  "dnssec": "false",
  "error": "",
  "scrapedAt": "2026-11-10T12:00:00.000Z"
}

---

📊 Output field coverage & reference

🌐 Core DNS & Resolution Fields

Field	Type	Coverage	Description
domain	string	🟢 High (100%)	The normalized, lowercased domain that was checked.
resolves	string	🟢 High (100%)	"true" if the domain has at least one valid A or AAAA record.
aRecords	string	🟢 High (99%)	Comma-separated list of IPv4 addresses.
aaaaRecords	string	🟡 Conditional (40%)	Comma-separated list of IPv6 addresses (if configured).
nsRecords	string	🟢 High (99%)	Authoritative nameservers serving the domain zone.
txtRecords	string	🟢 High (98%)	Concatenated TXT records (delimited with |).

✉️ Email Authentication & Provider Fields

Field	Type	Coverage	Description
mxRecords	string	🟢 High (97%)	Mail exchange servers sorted by priority ascending.
mailProvider	string	🟢 High (76%)	Detected mail vendor (Google Workspace, Microsoft 365, Zoho, etc.).
hasSpf	string	🟢 High (100%)	"true" if an SPF record (v=spf1) is found in TXT records.
hasDmarc	string	🟢 High (100%)	"true" if a DMARC record (v=dmarc1) exists on _dmarc.<domain>.
hasDkim	string	🟡 Conditional (20%)	"true" if DKIM indicators are visible on the root TXT record.

🏢 Registry, WHOIS & Security Fields

Field	Type	Coverage	Description
registrar	string	🟢 High (85% on RDAP TLDs)	Name of the ICANN-accredited domain registrar.
createdDate	string	🟢 High (85% on RDAP TLDs)	Original domain registration date (YYYY-MM-DD).
updatedDate	string	🟢 High (85% on RDAP TLDs)	Last time the registry record was modified.
expiryDate	string	🟢 High (85% on RDAP TLDs)	Domain expiration / renewal date (YYYY-MM-DD).
daysUntilExpiry	string	🟢 High (85% on RDAP TLDs)	Integer number of days remaining until expiration.
statusCodes	string	🟢 High (80% on RDAP TLDs)	EPP status codes (e.g., clientTransferProhibited).
dnssec	string	🟢 High (85% on RDAP TLDs)	Delegation signing status ("true" / "false").
error	string	🟡 Conditional	Diagnostic error string if the domain failed to resolve.
scrapedAt	string	🟢 High (100%)	ISO 8601 timestamp when the check was performed.

---

🔁 Pagination & scale

There is no website pagination here — this is a bulk batch processing Actor. You supply the full list of domains and the Actor fans them out across an optimized worker pool governed by maxConcurrency (1–200), checking many at once and streaming clean, de-duplicated records directly to the Apify dataset.

Input domains are cleaned, trimmed, lowercased, and de-duplicated before execution begins. A list containing duplicate URLs, emails, or protocol variants collapses into unique domain entries, ensuring 0 duplicate rows in your output.

Scale is governed by your input size and the maxItems safety cap:

• Set maxItems: 0 to check all provided domains without limit.
• Set maxItems: 1000+ to safely process large enterprise domain batches.
• DNS-only mode (whoisLookup: false) achieves peak throughput (>1,000 domains/min).
• Full WHOIS lookups utilize an adaptive token-bucket queue mapped directly to official IANA registry servers to prevent 429 throttling.

---

⚙️ How to use

1. Open the Actor on Apify Console and click Try for free / Start.
2. Paste your domains (one per line) — or full URLs or email addresses; the domain is extracted automatically.
3. Toggle Include WHOIS on for registrar/expiry/status, or off for a faster DNS-only run.
4. (Optional) Set Max domains and Speed (concurrency).
5. Click Save & Run, then export as JSON, CSV or Excel, or pull the dataset via the Apify API.
6. Automate it: schedule runs weekly for ongoing monitoring, or wire the dataset into Make, Zapier, Google Sheets or webhooks.

---

💻 API & Integration examples

cURL

curl -X POST "https://api.apify.com/v2/acts/haketa~bulk-dns-checker/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "domains": ["github.com", "stripe.com", "shopify.com"],
    "whoisLookup": true,
    "maxConcurrency": 50
  }'

Python (Apify Client)

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "domains": ["github.com", "google.com", "cloudflare.com"],
    "whoisLookup": True,
    "maxConcurrency": 50,
    "maxItems": 0
}

run = client.actor("haketa/bulk-dns-checker").call(run_input=run_input)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['domain']}: Mail={item['mailProvider']}, Expires={item['expiryDate']} ({item['daysUntilExpiry']} days)")

Node.js (Apify Client)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('haketa/bulk-dns-checker').call({
    domains: ['github.com', 'google.com', 'cloudflare.com'],
    whoisLookup: true,
    maxConcurrency: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Audited ${items.length} domains successfully:`);
console.table(items, ['domain', 'resolves', 'mailProvider', 'hasSpf', 'hasDmarc', 'registrar']);

---

💡 Pro tips for maximum speed & quality

• Disable whoisLookup for massive lists: When auditing 10,000+ domains for email deliverability or DNS resolution only, set whoisLookup: false. Lookups finish up to 10x faster.
• Sort by daysUntilExpiry ascending: Quickly identify upcoming renewals across your domain portfolio to prevent accidental expirations.
• Filter hasSpf: false or hasDmarc: false: Instantly generate remediation checklists for cold email infrastructure and domain reputation audits.
• Check statusCodes for transfer security: Unlocked production domains (missing clientTransferProhibited) represent significant hijacking risks.
• Segment lists by mailProvider: Build targeted outreach campaigns segmented by corporate email infrastructure (e.g., Microsoft 365 vs Google Workspace).

---

❓ Frequently Asked Questions

Is running bulk DNS and WHOIS checks legal?
Yes. This Actor reads exclusively public DNS resource records and public RDAP (ICANN registry) data for the domains you specify. These are the identical public records returned by standard tools like dig and command-line WHOIS.

What specific records does the DNS checker look up?
It resolves A, AAAA, MX, NS, and TXT records, along with dedicated SPF, DMARC, and DKIM indicators. With WHOIS enabled, it enriches the result with registrar name, creation date, modification date, expiry date, days until expiry, EPP status codes, and DNSSEC validation.

How many domains can it handle in a single run?
From a single domain to hundreds of thousands of domains. Concurrency scales up to 200 parallel workers, processing large lists quickly and reliably without platform timeouts.

Do I need proxies or an account login?
No. DNS and RDAP are public protocols that do not require credentials, third-party API keys, or proxy configurations. The Actor connects directly to public nameservers and authoritative registry servers.

Why are registrar and expiry fields occasionally empty?
These fields rely on the RDAP protocol. While virtually all generic top-level domains (.com, .net, .org, .io, etc.) support RDAP, some legacy country-code TLDs (ccTLDs) or restricted extensions (.edu) do not publish public RDAP endpoints. DNS resolution records are always fully populated regardless of RDAP availability.

Can I submit full URLs or email addresses instead of bare domains?
Yes! Formats like https://subdomain.company.com/landing?ref=123 or sales@company.com are automatically cleaned and normalized down to bare domain names (subdomain.company.com or company.com), removing duplicates seamlessly.

How does mail provider detection work?
The scraper inspects MX exchanger host patterns using fingerprint matching (e.g., *.protection.outlook.com maps to Microsoft 365, aspmx.l.google.com maps to Google Workspace), supporting Proofpoint, Mimecast, Amazon SES, Mailgun, SendGrid, Zoho, Proton, and others.

How fresh is the scraped data?
All queries execute live in real time against authoritative nameservers and registry databases during the run. Each record includes a precise scrapedAt timestamp.

Can I run this on a scheduled recurring cron?
Yes! On the Apify platform, you can configure a schedule (e.g., weekly or monthly) to automatically re-check your domain watchlist and pipe the newly generated dataset into your database or Google Sheets via webhooks.

How does this compare to running dig or host on the command line?
dig and host are single-domain command-line utilities. They don't handle thousands of domains concurrently, don't perform automatic URL/email sanitization, don't query authoritative RDAP servers for registrar/expiration dates, and don't export clean JSON/CSV datasets ready for business intelligence tools.

Why does hasDkim occasionally return false when DKIM is enabled?
DKIM public keys are typically published on selector-specific subdomains (such as google._domainkey.example.com or k1._domainkey.example.com), rather than on the root apex domain TXT record. This actor flags root TXT DKIM references, but selector-specific records require prior knowledge of the selector name.

What does resolves: "false" mean when MX or NS records exist?
The resolves flag specifically indicates whether the root apex domain has an active A (IPv4) or AAAA (IPv6) web host record. A domain might have no website running (no apex A record) yet still maintain active email routing (MX) and nameservers (NS).

---

🏗️ Architecture & High-Throughput Engine

To provide maximum throughput without getting blocked or bottlenecked, the checker splits workloads into two specialized worker pools:

1. High-Speed DNS Worker Pool:

   • Handles A, AAAA, MX, NS, TXT, and _dmarc lookups.
   • Operates at high concurrency (up to 200 workers) with low per-query latency (~50-150ms).
   • Resolves via native OS DNS with strict individual timeouts to prevent slow domains from blocking execution.

2. Adaptive RDAP Registry Pool:

   • Resolves authoritative registry endpoints dynamically from IANA bootstrap records.
   • Throttles concurrency independently to honor ICANN rate expectations.
   • Implements automated exponential backoff retry logic upon encountering HTTP 429 or 5xx responses.

---

🔔 Webhook & Automation Recipes

Integrate your DNS audits directly into operational alerting pipelines:

• Slack / Discord Expiration Alerts: Connect an Apify Webhook that triggers on run completion. Filter for any domains with daysUntilExpiry < 30 and send an instant notification to your team channel.
• Zapier / Make CRM Sync: Automatically trigger DNS enrichment when a new enterprise company domain is entered into HubSpot or Salesforce.
• Google Sheets Live Sync: Pipe new audit runs into an executive reporting spreadsheet for IT compliance reviews.

---

🔗 Related Scrapers & Data Tools

• Bulk Company Domain Finder
• Impressum / Legal Notice Extractor
• BBB Business Scraper
• Marktplaats Scraper
• Kijiji Scraper
• ZAP Imóveis Scraper

---

📋 Changelog

v1.2 (2026-11)

• Unlocked 1,000+ domain bulk batching: Eliminated artificial caps and optimized worker pooling for continuous deep batch runs.
• Enhanced demo fallback registry: Expanded built-in sample list to over 50 diverse domains for rich empty/demo runs.
• Authoritative RDAP resolution: Upgraded IANA bootstrap caching and separate RDAP worker queue with exponential backoff on 429 rate limits.
• Field normalization: Standardized clean ISO date formatting, normalized boolean string representations, and zero-duplicate dataset outputs.

v1.1 (2026-08)

• Added automatic URL and email normalization to bare domains.
• Integrated DMARC detection via _dmarc.<domain> queries.
• Added MX-to-mailProvider fingerprint mapping for major enterprise email providers.

v1.0 (2026-07)

• Initial release featuring parallel DNS record lookups (A, AAAA, MX, NS, TXT) and basic RDAP WHOIS enrichment.