# Domain Intelligence Scraper | $10/1K | WHOIS+DNS+SSL (`apivault_labs/domain-intelligence-scraper`) Actor

WHOIS + DNS + SSL + subdomain lookup for any domain. Perfect for OSINT, security audits, SEO research, lead generation.

- **URL**: https://apify.com/apivault\_labs/domain-intelligence-scraper.md
- **Developed by:** [Apivault Labs](https://apify.com/apivault_labs) (community)
- **Categories:** Developer tools, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 domain analyzeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## 🌐 Domain Intelligence Scraper | $10/1K | WHOIS + DNS + SSL + Subdomains

Bulk domain intelligence in one actor. Get **WHOIS** (registrar, dates, owner), **DNS records** (A, MX, NS, TXT, CAA), **SSL certificate history**, **subdomain discovery**, and **HTTP security headers** — all from a single API call. Perfect for cybersecurity, OSINT, SEO, and B2B enrichment.

### ✨ Key Features

- 📋 **WHOIS** via RDAP (modern, JSON, fast)
- 🌐 **DNS records** via Cloudflare DNS-over-HTTPS (A, AAAA, MX, NS, TXT, CNAME, CAA)
- 🔒 **SSL certificate history** via crt.sh (certificate transparency logs)
- 🕸️ **Subdomain discovery** from SSL certificate SANs (passive OSINT)
- 🛡️ **HTTP security headers** (HSTS, CSP, X-Frame-Options, Cloudflare/CloudFront detection)
- 📊 **Bulk** — process 100s of domains in parallel
- 🔒 **Zero API keys** — all public data sources

### Input

#### Single domain deep dive

```json
{
  "domains": ["apify.com"]
}
````

#### Bulk security audit

```json
{
  "domains": ["company.com", "subsidiary.io", "legacy.net"],
  "extractSubdomains": true,
  "maxConcurrency": 10
}
```

#### DNS + WHOIS only (fast mode)

```json
{
  "domains": ["lead1.com", "lead2.com", "lead3.com"],
  "extractWhois": true,
  "extractDns": true,
  "extractSsl": false,
  "extractSubdomains": false,
  "extractHttp": false
}
```

#### Input Parameters

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `domains` | string\[] | ✅ | Domains (URLs or bare). Example: `apify.com` |
| `extractWhois` | bool | ❌ | WHOIS via RDAP (default: true) |
| `extractDns` | bool | ❌ | DNS records (default: true) |
| `extractSsl` | bool | ❌ | SSL certificate data (default: true) |
| `extractSubdomains` | bool | ❌ | Subdomain discovery via crt.sh (default: true) |
| `extractHttp` | bool | ❌ | HTTP response headers (default: true) |
| `maxConcurrency` | int | ❌ | Parallel lookups (default: 5) |
| `timeout` | int | ❌ | Timeout per API call (default: 15) |

### Output

```json
{
  "success": true,
  "domain": "apify.com",
  "whois": {
    "registrationDate": "2015-12-17T08:31:32Z",
    "expirationDate": "2028-12-17T08:31:32Z",
    "lastUpdatedDate": "2024-10-23T11:15:48Z",
    "registrar": "Gandi SAS",
    "registrantOrg": "Apify Technologies",
    "nameservers": ["ns1.digitalocean.com", "ns2.digitalocean.com"],
    "status": ["client transfer prohibited"]
  },
  "dns": {
    "a": ["18.184.171.89"],
    "aaaa": [],
    "mx": ["10 aspmx.l.google.com.", "20 alt1.aspmx.l.google.com."],
    "ns": ["ns1.digitalocean.com.", "ns2.digitalocean.com."],
    "txt": ["v=spf1 include:_spf.google.com ~all", "google-site-verification=..."],
    "cname": [],
    "caa": ["0 issue \"letsencrypt.org\""]
  },
  "ssl": {
    "certCount": 847,
    "latestIssuer": "C=US, O=Let's Encrypt, CN=R10",
    "latestNotBefore": "2026-04-15T10:23:00Z",
    "latestNotAfter": "2026-07-14T10:23:00Z",
    "latestCommonName": "apify.com"
  },
  "subdomains": ["api.apify.com", "console.apify.com", "docs.apify.com", ...],
  "subdomainCount": 47,
  "http": {
    "finalUrl": "https://apify.com/",
    "statusCode": 200,
    "server": "cloudflare",
    "xPoweredBy": "",
    "cfRay": "8f9a...abcd-FRA",
    "strictTransportSecurity": "max-age=63072000; includeSubDomains",
    "contentSecurityPolicy": "default-src 'self'...",
    "viaCloudflare": true,
    "viaCloudfront": false,
    "redirected": true
  }
}
```

### Use Cases

#### 🛡️ Cybersecurity Auditing

- Detect missing HSTS / CSP headers across your domains
- Find forgotten subdomains (attack surface mapping)
- Discover expiring SSL certificates
- Bulk-audit a portfolio after M\&A

#### 🔍 OSINT Investigations

- Map an organization's entire domain infrastructure
- Find associated subsidiaries via WHOIS registrant
- Track nameserver patterns (Cloudflare, Route53, self-hosted)

#### 📈 SEO Research

- Analyze competitor DNS setup (CDN used, MX = which email provider)
- Find expiring domains in your niche
- Detect migration patterns (nameserver changes)

#### 💼 B2B Lead Enrichment

- Get company tech stack signals (Cloudflare, CloudFront, custom server)
- Identify email provider (MX record → Google, Microsoft, etc.)
- Estimate company size via subdomain count

#### 🧬 Bulk Domain Audits

- Feed a list of domains, get a full profile of each
- Diff over time to spot changes

### Pricing

- **$0.01 per domain** ($10 per 1,000 domains)
- All 5 data sources included in one price
- Pay only for successfully processed domains

### How it works

Five independent API calls per domain, all to **free public services**:

1. **RDAP** (`rdap.org`) — modern WHOIS, JSON format, no rate limit for normal use
2. **Cloudflare DoH** (`cloudflare-dns.com/dns-query`) — DNS-over-HTTPS, fast and reliable
3. **crt.sh** — public certificate transparency log, sometimes slow but free
4. **HTTPS HEAD request** — lightweight, just response headers
5. **crt.sh SAN extraction** — subdomains from SSL cert history

No proxies needed. No API keys. No rate limit surprises.

### Notes

- **WHOIS privacy**: many registrars hide owner contact (GDPR). You'll get registrar + dates but often not registrant name.
- **crt.sh can be slow** — during peak hours, subdomain queries may take 10-20 seconds.
- **Subdomain discovery is passive** — you only find subdomains that appeared in SSL certs. Not a full port scan.
- **DNS** reflects what public DNS servers cache, not the authoritative zone.

### Pro tips

- **Security audits**: check `strictTransportSecurity` and `contentSecurityPolicy` across your assets
- **Lead enrichment**: `mx` containing `aspmx.l.google.com` = Google Workspace customer
- **Tech stack**: `server: cloudflare` + `cf-ray` header = hosted behind Cloudflare
- **M\&A due diligence**: run on both buyer and target domains to spot subsidiaries
- Combine with our **Shopify/WooCommerce scrapers** to validate e-commerce tech stack of target companies

# Actor input Schema

## `domains` (type: `array`):

List of domain names. Accepts full URLs or bare domains:
• example.com
• https://example.com
• www.example.com

## `extractWhois` (type: `boolean`):

Registrar, creation date, expiration, owner (RDAP)

## `extractDns` (type: `boolean`):

A, AAAA, MX, NS, TXT, CNAME records via DNS-over-HTTPS

## `extractSsl` (type: `boolean`):

SSL certificate chain from crt.sh transparency logs

## `extractSubdomains` (type: `boolean`):

Discover subdomains from SSL certificate history

## `extractHttp` (type: `boolean`):

Fetch https://domain/ and record response headers (server, security headers, redirects)

## `maxConcurrency` (type: `integer`):

Parallel domain lookups (default: 5)

## `timeout` (type: `integer`):

Timeout per individual API call

## Actor input object example

```json
{
  "domains": [
    "apify.com",
    "github.com"
  ],
  "extractWhois": true,
  "extractDns": true,
  "extractSsl": true,
  "extractSubdomains": true,
  "extractHttp": true,
  "maxConcurrency": 5,
  "timeout": 15
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "apify.com",
        "github.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("apivault_labs/domain-intelligence-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "apify.com",
        "github.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("apivault_labs/domain-intelligence-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "apify.com",
    "github.com"
  ]
}' |
apify call apivault_labs/domain-intelligence-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=apivault_labs/domain-intelligence-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Domain Intelligence Scraper | $10/1K | WHOIS+DNS+SSL",
        "description": "WHOIS + DNS + SSL + subdomain lookup for any domain. Perfect for OSINT, security audits, SEO research, lead generation.",
        "version": "1.0",
        "x-build-id": "vdedRcgNqgtSByJUw"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/apivault_labs~domain-intelligence-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-apivault_labs-domain-intelligence-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/apivault_labs~domain-intelligence-scraper/runs": {
            "post": {
                "operationId": "runs-sync-apivault_labs-domain-intelligence-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/apivault_labs~domain-intelligence-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-apivault_labs-domain-intelligence-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "domains"
                ],
                "properties": {
                    "domains": {
                        "title": "Domains to analyze",
                        "type": "array",
                        "description": "List of domain names. Accepts full URLs or bare domains:\n• example.com\n• https://example.com\n• www.example.com",
                        "items": {
                            "type": "string"
                        }
                    },
                    "extractWhois": {
                        "title": "WHOIS Data",
                        "type": "boolean",
                        "description": "Registrar, creation date, expiration, owner (RDAP)",
                        "default": true
                    },
                    "extractDns": {
                        "title": "DNS Records",
                        "type": "boolean",
                        "description": "A, AAAA, MX, NS, TXT, CNAME records via DNS-over-HTTPS",
                        "default": true
                    },
                    "extractSsl": {
                        "title": "SSL Certificates",
                        "type": "boolean",
                        "description": "SSL certificate chain from crt.sh transparency logs",
                        "default": true
                    },
                    "extractSubdomains": {
                        "title": "Subdomains (via crt.sh)",
                        "type": "boolean",
                        "description": "Discover subdomains from SSL certificate history",
                        "default": true
                    },
                    "extractHttp": {
                        "title": "HTTP Response Headers",
                        "type": "boolean",
                        "description": "Fetch https://domain/ and record response headers (server, security headers, redirects)",
                        "default": true
                    },
                    "maxConcurrency": {
                        "title": "Max Concurrency",
                        "minimum": 1,
                        "maximum": 20,
                        "type": "integer",
                        "description": "Parallel domain lookups (default: 5)",
                        "default": 5
                    },
                    "timeout": {
                        "title": "Timeout per lookup (seconds)",
                        "minimum": 5,
                        "maximum": 60,
                        "type": "integer",
                        "description": "Timeout per individual API call",
                        "default": 15
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```