Domain Intelligence Suite

Apify Actor for comprehensive domain intelligence gathering.

Runs four independent intelligence modules against any domain name — WHOIS records, DNS resolution, SSL/TLS certificate inspection, and subdomain discovery via Certificate Transparency logs. Each module handles errors gracefully; if one fails, the others still deliver results.

---

Modules

Module	Source	Key Data
WHOIS	python-whois	Registrar, dates, nameservers, status, contacts
DNS	dnspython	A, AAAA, MX, NS, TXT, CNAME, SOA records
SSL	ssl + socket	Issuer, expiry, days remaining, SANs, serial
Subdomains	crt.sh	Unique subdomains from CT logs, deduplicated

---

Quick Start

Run on Apify

1. Go to the Actor page on Apify Console
2. Click Try for free
3. Enter a domain name (e.g. apify.com)
4. Click Run

Run Locally

# Install Apify CLI
npm install -g apify-cli

# Clone and run
git clone <this-repo>
cd domain-intelligence-suite

# Install dependencies
pip install -r requirements.txt

# Run
apify run

Run via API

curl "https://api.apify.com/v2/acts/<username>/domain-intelligence-suite/runs" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domain": "example.com"}'

---

Input

Field	Type	Default	Description
domain (required)	string	—	Domain to investigate (e.g. example.com). Strips protocol/path automatically.
modules	string[]	All four	Which modules to run: whois, dns, ssl, subdomains
sslPort	integer	443	Port for SSL certificate inspection
maxSubdomains	integer	500	Max subdomains returned from crt.sh (10–2000)
dnsRecordTypes	string[]	All seven	DNS record types: A, AAAA, MX, NS, TXT, CNAME, SOA
includeWildcardSubdomains	boolean	true	Include *.example.com wildcards in subdomain results

Example Input

{
  "domain": "github.com",
  "modules": ["whois", "ssl"],
  "sslPort": 443,
  "maxSubdomains": 200
}

---

Output

Results are pushed to the default dataset as JSON. Each run produces one item:

{
  "domain": "example.com",
  "timestamp": "2026-07-07T12:00:00+00:00",
  "whois": {
    "domain": "example.com",
    "queried_at": "2026-07-07T12:00:00.123456",
    "data": {
      "domain_name": "EXAMPLE.COM",
      "registrar": "RESERVED-Internet Assigned Numbers Authority",
      "creation_date": "1995-08-14T04:00:00+00:00",
      "expiration_date": "2027-08-13T04:00:00+00:00",
      "name_servers": ["A.IANA-SERVERS.NET", "B.IANA-SERVERS.NET"],
      "status": ["serverDeleteProhibited", "serverTransferProhibited"]
    }
  },
  "dns": {
    "domain": "example.com",
    "queried_at": "2026-07-07T12:00:00.456789",
    "records": {
      "A": ["93.184.216.34"],
      "AAAA": ["2606:2800:220:1:248:1893:25c8:1946"],
      "MX": [
        {"preference": 0, "exchange": ""}
      ],
      "NS": ["a.iana-servers.net", "b.iana-servers.net"],
      "TXT": ["v=spf1 -all", "wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn"],
      "SOA": [
        {
          "mname": "a.iana-servers.net",
          "rname": "nstld.verisign-grs.com",
          "serial": 2026070444,
          "refresh": 1800,
          "retry": 900,
          "expire": 604800,
          "minimum": 86400
        }
      ]
    }
  },
  "ssl": {
    "domain": "example.com",
    "port": 443,
    "queried_at": "2026-07-07T12:00:01.123456",
    "certificate": {
      "issuer": "C=US, O=DigiCert Inc, CN=DigiCert TLS RSA SHA256 2020 CA1",
      "subject": "C=US, ST=California, L=Los Angeles, O=Internet Corporation for Assigned Names and Numbers, CN=www.example.org",
      "valid_from": "2026-01-30T00:00:00+00:00",
      "valid_to": "2027-03-01T23:59:59+00:00",
      "days_remaining": 237,
      "expired": false,
      "subject_alt_names": ["www.example.org", "example.org", "example.com"],
      "serial_number": "0FBE...",
      "version": 3
    }
  },
  "subdomains": {
    "domain": "example.com",
    "source": "crt.sh",
    "queried_at": "2026-07-07T12:00:02.789012",
    "subdomains": [
      "dev.example.com",
      "mail.example.com",
      "staging.example.com",
      "www.example.com"
    ],
    "total_found": 4,
    "returned": 4
  }
}

Error Handling

When a module encounters an error, it returns an "error" field instead of data — the Actor never crashes:

{
  "whois": {
    "domain": "nonexistent-xyz123.com",
    "queried_at": "2026-07-07T12:00:00",
    "error": "WHOIS lookup failed (domain may not exist): No match for domain..."
  }
}

---

Error Resilience

The Actor is designed to handle failures at every level:

Scenario	Behaviour
Invalid/non-existent domain	Module returns error field with description
DNS resolution failure	NXDOMAIN / NoNameservers / Timeout per record type
SSL connection refused	error on ssl module; other modules unaffected
SSL timeout	Clear timeout message
crt.sh rate limit (HTTP 429)	Captured as error
crt.sh unavailable	Timeout + error message
Module crash (unhandled)	Top-level try/catch in main.py catches it

---

Docker

# Build
docker build -t domain-intelligence-suite .

# Run locally
docker run --rm -e APIFY_TOKEN=dummy \
  -e APIFY_INPUT='{"domain": "example.com"}' \
  domain-intelligence-suite

For production, the Actor runs on Apify's managed infrastructure — no Docker setup needed.

---

Development

Project Structure

domain-intelligence-suite/
├── .actor/
│   └── actor.json          # Apify actor spec
├── src/
│   ├── __init__.py
│   ├── main.py             # Entry point — Actor.run()
│   └── modules/
│       ├── __init__.py
│       ├── whois.py         # WHOIS lookup (python-whois)
│       ├── dns.py           # DNS resolver (dnspython)
│       ├── ssl_check.py     # SSL inspection (ssl + socket)
│       └── subdomains.py    # Subdomain enum (crt.sh API)
├── INPUT_SCHEMA.json        # Input definition for Apify Console
├── dataset_schema.json      # Output schema
├── Dockerfile
├── requirements.txt
└── README.md

Adding a New Module

1. Create src/modules/<name>.py with an async def lookup_*(domain) function
2. Return {"data": ..., "error": ...} pattern
3. Register in src/main.py → MODULE_NAMES + task creation
4. Add to INPUT_SCHEMA.json modules enum

Local Testing

# Set mock input and run
APIFY_INPUT='{"domain": "example.com", "modules": ["dns"]}' python -m src.main

---

Pricing

• Pay-per-event model — you're only charged for Actor compute time
• Typical run: 3–10 seconds depending on DNS propagation and crt.sh response
• Free tier: 10,000+ free runs/month on Apify

---

License

MIT