# Domain Scraper — WHOIS, DNS, Expiry & Availability (`glitchbound/domain-scraper`) Actor

Bulk domain lookup: registrar, registration and expiry dates, lock status, DNSSEC, nameservers and live DNS records (A, MX, NS, TXT, CAA). Detects the mail and DNS provider, checks SPF/DMARC, and tells you whether a domain is available. Official RDAP and DNS — no key.

- **URL**: https://apify.com/glitchbound/domain-scraper.md
- **Developed by:** [Daniel Meshulam](https://apify.com/glitchbound) (community)
- **Categories:** SEO tools, Lead generation, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 domains

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Domain Scraper — WHOIS, DNS, Expiry & Availability

Bulk domain lookup that returns something you can **act on**, not a wall of raw WHOIS text.

For every domain: who registers it, when it expires, whether it's locked, and — from live DNS — **which mail provider and DNS provider it actually runs on**, plus SPF/DMARC status.

Built on RDAP (the IETF's official WHOIS replacement, RFC 9083) and DNS-over-HTTPS. No key, no login.

### What you get

| Field | Example |
|---|---|
| `domain`, `isRegistered`, `isAvailable` | `stripe.com`, `true`, `false` |
| `registrar` | `SafeNames Ltd.` |
| `registeredAt`, `expiresAt`, `updatedAt` | |
| **`daysUntilExpiry`** | `410` |
| **`ageYears`** | `30.9` |
| **`mailProvider`** | `google` · `microsoft-365` · `zoho` · `proton` … |
| **`dnsProvider`** | `cloudflare` · `aws-route53` · `azure` · `vercel` … |
| `hasSPF`, `hasDMARC`, `dmarcPolicy` | `true`, `true`, `reject` |
| `isLocked`, `dnssec`, `statusCodes` | `true`, `true` |
| `nameservers`, `dnsA`, `dnsMX`, `dnsNS`, `dnsTXT`, `dnsCAA` | |
| `verificationTokens` | the `*-site-verification` TXT records |

### Example input

```json
{
  "domains": ["github.com", "stripe.com", "https://openai.com/about"],
  "includeDns": true,
  "recordTypes": ["A", "MX", "NS", "TXT"],
  "includeEmailSecurity": true
}
```

Full URLs and email addresses are accepted — `https://example.com/pricing` and `sales@example.com` both reduce to `example.com`.

### Built-in filters

**`onlyRegistered: true`** — drop available domains. Useful when you're feeding in a scraped list.

**`expiringWithinDays: 60`** — only domains expiring soon. Point it at a portfolio for renewal alerts, or at a wishlist for drop-catching.

Filtered-out domains are **not charged**.

### Availability check

An unregistered domain returns `isAvailable: true`. This comes from the registry's own RDAP server returning "no such object" — it's authoritative, not a reseller's guess. Available domains skip the DNS lookups entirely, so bulk availability checks are fast and cheap.

### Who this is for

- **Agencies & sales teams** — qualify leads by tech stack: who's on Google Workspace, who's on Microsoft 365, who has no DMARC
- **Domain investors** — expiry monitoring across a portfolio, availability checks in bulk
- **Security & IT** — SPF/DMARC audit across every domain you own
- **SEO** — domain age and registrar for competitor and backlink analysis

### Privacy

This Actor returns **no registrant personal data**. Post-GDPR, RDAP exposes only the registrar for virtually every TLD, and no attempt is made to recover the individual behind a domain from any other source. What comes back is operational: registrar, dates, status, and where mail and DNS are hosted.

### Pricing

Pay per result. One row = one domain. Invalid input and lookup failures come back as `error` rows and are **not** charged, and neither are domains removed by your filters.

### Notes

- `includeEmailSecurity` costs one extra DNS lookup per domain (DMARC lives at `_dmarc.<domain>`, not on the apex).
- Some ccTLDs run thin RDAP servers and return fewer fields — you'll get whatever the registry publishes rather than an error.
- Set a **max charge per run** in the run options for a hard spend ceiling — the Actor honors it and stops cleanly.

### FAQ

#### Do I need a WHOIS or DNS API key?

No. This uses RDAP, the IETF's official WHOIS replacement (RFC 9083), plus
DNS-over-HTTPS. Both are open and key-free.

#### Can I check whether a domain is available to register?

Yes. An unregistered domain returns `isAvailable: true`, and that comes from the
registry's own RDAP server answering "no such object", so it is authoritative
rather than a reseller's guess. Available domains skip DNS lookups entirely, so bulk
availability checks are fast and cheap.

#### Will this give me the registrant's name and email?

No, and not because of a limitation here. Post-GDPR, RDAP exposes only the
registrar for virtually every TLD. No attempt is made to recover the individual
behind a domain from any other source. What you get is operational: registrar,
dates, status, and where mail and DNS are hosted.

#### How do I find out which email provider a company uses?

`mailProvider` is derived from live MX records and resolves to names like
`google`, `microsoft-365`, `zoho` or `proton`. `dnsProvider` does the same for
nameservers: `cloudflare`, `aws-route53`, `azure`, `vercel`. That is the field
sales teams actually want out of a WHOIS tool.

#### Can I audit SPF and DMARC across all my domains?

Yes. `includeEmailSecurity` returns `hasSPF`, `hasDMARC` and `dmarcPolicy` (the
`p=` value, so you can tell `none` from `reject`). It costs one extra DNS lookup
per domain because DMARC lives at `_dmarc.<domain>`, not on the apex.

#### Can I get renewal alerts for a domain portfolio?

Set `expiringWithinDays: 60` and schedule it. Only domains inside that window are
returned, and the ones filtered out are not charged.

#### Do I have to clean my input list first?

No. Full URLs and email addresses are accepted, so `https://example.com/pricing`
and `sales@example.com` both reduce to `example.com`.

#### Why do some domains return fewer fields?

Some ccTLDs run thin RDAP servers. You get whatever the registry publishes rather
than an error.

# Actor input Schema

## `domains` (type: `array`):

Domains to look up, one per line. Full URLs (https://example.com/page) and email addresses (a@example.com) are accepted and reduced to the domain automatically.

## `includeDns` (type: `boolean`):

Look up live DNS records for each domain and derive the mail and DNS provider from them.

## `recordTypes` (type: `array`):

Which records to fetch. Valid: A, AAAA, MX, NS, TXT, CNAME, SOA, CAA.

## `includeEmailSecurity` (type: `boolean`):

Check SPF and DMARC and report the DMARC policy. Costs one extra lookup per domain.

## `onlyRegistered` (type: `boolean`):

Skip domains that are available (unregistered). Filtered-out domains are not charged.

## `expiringWithinDays` (type: `integer`):

Only return domains expiring within this many days. 0 = no filter. Useful for renewal and drop-catch monitoring.

## `proxyConfiguration` (type: `object`):

Proxy settings.

## Actor input object example

```json
{
  "domains": [
    "github.com",
    "stripe.com"
  ],
  "includeDns": true,
  "recordTypes": [
    "A",
    "MX",
    "NS",
    "TXT"
  ],
  "includeEmailSecurity": true,
  "onlyRegistered": false,
  "expiringWithinDays": 0,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Registration, expiry, DNS records and email-security posture per domain.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "github.com",
        "stripe.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("glitchbound/domain-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "github.com",
        "stripe.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("glitchbound/domain-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "github.com",
    "stripe.com"
  ]
}' |
apify call glitchbound/domain-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=glitchbound/domain-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/ES6QUWwxuwkPRyyIu/builds/cNa1TPGdG6Rs6UQYO/openapi.json