# Domain WHOIS & DNS Intelligence Scraper (`ninhothedev/domain-intel-scraper`) Actor

$1/1K 🔥 Domain intelligence scraper! WHOIS, DNS, SPF/DMARC & hosting provider in one row. No key. JSON, CSV, Excel or API in seconds. Enrich leads & audit domain security ⚡

- **URL**: https://apify.com/ninhothedev/domain-intel-scraper.md
- **Developed by:** [ninhothedev](https://apify.com/ninhothedev) (community)
- **Categories:** Developer tools, Lead generation, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Domain WHOIS & DNS Intelligence Scraper

**One row per domain: WHOIS/RDAP registration data, the full DNS zone, mail security posture (SPF, DMARC, DNSSEC) and the detected DNS + mail provider — combined, normalised, and ready for your CRM or spreadsheet. No API key, no account, no proxy required.**

Paste a list of domains, get back a complete technical and administrative profile of each one. The actor talks to the authoritative RDAP registry (via the official IANA bootstrap at `rdap.org`) and to Google Public DNS over HTTPS, then merges both worlds into a single flat record.

***

### How this differs from my two single-purpose actors

I already publish two focused, cheaper actors. This one is the **combined** product — use it when you want the whole picture in one row instead of joining two datasets yourself.

| | [RDAP Domain Scraper](https://apify.com/ninhothedev/rdap-domain-scraper) | [DNS Records Scraper](https://apify.com/ninhothedev/dns-records-scraper) | **Domain Intel Scraper (this one)** |
|---|---|---|---|
| Registrar, creation & expiry dates | Yes | No | **Yes** |
| Domain status codes, DNSSEC | Yes | No | **Yes** |
| A / AAAA / MX / NS / TXT / SOA | No | Yes | **Yes** |
| SPF record + `has_spf` flag | No | Partly (raw TXT only) | **Yes, parsed** |
| DMARC record + `p=` policy | No | No | **Yes, parsed** |
| Nameserver provider detection | No | No | **Yes (60+ providers)** |
| Mail provider detection from MX | No | No | **Yes (35+ providers)** |
| Computed `age_days` / `days_until_expiry` | No | No | **Yes** |
| Output shape | RDAP-only row | DNS-only row | **One enriched profile row** |

If you only need WHOIS data, use the RDAP actor. If you only need raw DNS answers, use the DNS actor. If you need a **domain intelligence profile** — registration + infrastructure + email security in one line — use this one.

***

### What you get per domain

```json
{
  "domain": "stripe.com",
  "registrar": "SafeNames Ltd.",
  "registered_at": "1995-09-12T04:00:00Z",
  "expires_at": "2027-09-11T04:00:00Z",
  "updated_at": "2025-10-01T01:39:51Z",
  "age_days": 11277,
  "days_until_expiry": 409,
  "status": ["client delete prohibited", "client transfer prohibited"],
  "dnssec": false,
  "nameservers": ["ns-1087.awsdns-07.org", "ns-423.awsdns-52.com"],
  "ns_provider": "AWS Route 53",
  "a_records": ["198.202.176.41"],
  "aaaa_records": [],
  "mx_records": ["10 aspmx.l.google.com", "20 alt1.aspmx.l.google.com"],
  "mx_provider": "Google Workspace",
  "txt_records": ["google-site-verification=...", "v=spf1 ip4:198.2.180.60/32 ..."],
  "spf": "v=spf1 ip4:198.2.180.60/32 include:spf1.stripe.com ~all",
  "has_spf": true,
  "dmarc": "v=DMARC1; p=reject; pct=100; rua=mailto:dmarc-reports@stripe.com;",
  "has_dmarc": true,
  "dmarc_policy": "reject",
  "soa": "ns-1882.awsdns-43.co.uk. awsdns-hostmaster.amazon.com. 1 900 900 1209600 3600",
  "ttl_a": 30,
  "source": "rdap+doh",
  "scraped_at": "2026-07-28T14:31:02Z"
}
```

Every field is nullable — a domain without MX records simply gets an empty list, a ccTLD without RDAP coverage still returns its complete DNS profile.

***

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | select | `domains` | Currently one mode: a domain list in, one intelligence row per domain out. |
| `domains` | array | `["stripe.com", "github.com"]` | Domains or full URLs. Scheme, `www.`, path, query and port are stripped automatically. |
| `maxItems` | integer | `100` (max `1000`) | Hard cap on processed domains, so cost stays predictable. |

```json
{
  "mode": "domains",
  "domains": ["stripe.com", "github.com", "https://www.shopify.com/pricing"],
  "maxItems": 100
}
```

***

### Use cases

- **Domain due diligence** — before you buy, partner with, or link to a domain: how old is it, who is the registrar, when does it expire, is DNSSEC on, is the registrar lock set?
- **Lead enrichment** — append the tech stack to your CRM. Knowing a prospect runs Google Workspace on Cloudflare DNS tells your sales team more than any firmographic field. Perfect for segmenting an outbound list by mail provider or hosting provider.
- **Security posture auditing** — find every domain in your portfolio (or your customer base) that is missing SPF, missing DMARC, or sitting on `p=none`. Those are the domains that can be spoofed today.
- **M\&A and portfolio research** — map the DNS and mail infrastructure of an acquisition target's whole domain estate in one run, and spot expiring domains before they lapse.
- **Expiry monitoring** — schedule the actor weekly and alert on `days_until_expiry < 30`.
- **Competitive intelligence** — see which CDN, DNS and email vendors your competitors migrated to, and when the zone last changed.

***

### Pricing

Roughly **$1 per 1,000 domains** in platform compute — the actor runs on 512 MB, uses no proxies, and issues only lightweight JSON requests (1 RDAP call + 7 DNS-over-HTTPS calls per domain). There is nothing to pay to a WHOIS API vendor on top.

### Speed & reliability

- Fully async, 10 domains resolved in parallel, pushed in batches of 50.
- `curl_cffi` with a Chrome 124 TLS fingerprint, with a standard-library `urllib` fallback.
- A single failing domain never kills the run; it is logged and skipped.
- The run fails loudly (`RuntimeError`) only if *no* domain could be resolved at all.

### Data source notes

- RDAP is the IETF successor to WHOIS and is served by the registries themselves. Coverage is excellent for gTLDs (`.com`, `.net`, `.org`, `.io`-style new gTLDs, `.dev`, `.app`) and for many ccTLDs. Some ccTLD registries (for example `.de`) do not publish an RDAP endpoint — those domains still get a complete DNS, SPF and DMARC profile, with the registration fields left `null`.
- Registrant contact details are redacted by registries under ICANN's GDPR-driven temporary specification. This actor therefore returns the **registrar**, not personal registrant data.
- DNS answers come from Google Public DNS (`dns.google`), so they reflect a public resolver's view of the zone.

### Related actors

- [RDAP Domain Scraper](https://apify.com/ninhothedev/rdap-domain-scraper) — registration data only, leanest option.
- [DNS Records Scraper](https://apify.com/ninhothedev/dns-records-scraper) — raw DNS record types only.
- [SSL Certificate Checker](https://apify.com/ninhothedev/ssl-certificate-checker) — issuer, validity window and expiry of the TLS certificate.
- [Security Headers Checker](https://apify.com/ninhothedev/security-headers-checker) — HSTS, CSP and the rest of the HTTP header posture.

Run all four over the same domain list for a complete external security and infrastructure profile.

### Local development

```bash
pip install -r requirements.txt
python -m src.main
python tests/test_smoke.py   # offline smoke tests, no network needed
```

### Legal

The actor reads only public registry (RDAP) and public DNS data. It performs no port scanning, no authentication and no intrusive probing.

# Actor input Schema

## `mode` (type: `string`):

What the actor should do. Currently only 'domains' is supported: every domain you pass in is turned into one enriched intelligence row (RDAP registration + DNS + mail posture).

## `domains` (type: `array`):

List of domains to look up, one per entry. Plain domains ('stripe.com') or full URLs ('https://www.stripe.com/pricing') both work - scheme, www., path and port are stripped automatically. Subdomains are queried as given.

## `maxItems` (type: `integer`):

Hard cap on how many domains are processed in this run. Extra domains beyond this number are ignored, which keeps cost predictable. Between 1 and 1000.

## Actor input object example

```json
{
  "mode": "domains",
  "domains": [
    "stripe.com",
    "github.com"
  ],
  "maxItems": 100
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "github.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("ninhothedev/domain-intel-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "stripe.com",
        "github.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("ninhothedev/domain-intel-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "github.com"
  ]
}' |
apify call ninhothedev/domain-intel-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=ninhothedev/domain-intel-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/dGCwv2FKd4M41mYJE/builds/oaB6fomYoMeaufRAX/openapi.json