# Extract Domains From Text (`automation-lab/domain-name-extractor`) Actor

Extract, normalize, deduplicate, and classify domains from pasted text, URL lists, and bounded public web pages.

- **URL**: https://apify.com/automation-lab/domain-name-extractor.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.39 / 1,000 domain extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Extract Domains From Text

Extract domains from text, URL lists, and bounded public web pages, then normalize and deduplicate the results into analysis-ready rows.

The Actor turns messy lead lists, logs, prose, email addresses, and page links into a consistent domain inventory. Each result includes the normalized hostname, registrable domain, public suffix, subdomain classification, occurrence count, and source lineage.

It runs entirely without a browser or login. Text and URL-list inputs are processed locally; optional page inputs use bounded anonymous HTTP requests.

### What this domain name extractor does

Use the Actor to:

- find domains inside pasted prose, logs, CSV text, and email addresses;
- normalize uppercase and internationalized domain names;
- merge `www` hostnames with their apex hostname when requested;
- keep meaningful subdomains such as `docs.apify.com` separate;
- identify the registrable domain and public suffix;
- deduplicate repeated occurrences while preserving their count;
- show which input route and source contained each hostname;
- optionally include and classify valid IPv4 addresses;
- scan the initial HTML or text response of supplied public pages.

The Actor does not perform DNS, WHOIS, or RDAP enrichment and does not crawl beyond each supplied page.

### Who is it for

**Lead operations teams** can clean mixed website columns before enrichment or outreach.

**Security analysts** can turn incident notes, logs, and threat reports into a unique hostname inventory.

**SEO and link analysts** can list domains and subdomains present in a page response.

**Data engineers** can normalize domain keys before joining datasets or sending records to another Actor.

**Developers** can replace one-off domain regex scripts with a reusable API endpoint and scheduled task.

### Why use this Actor

A regular expression can find domain-like strings, but it cannot reliably classify public suffixes such as `co.uk`, normalize IDNs, merge duplicates, or retain source lineage.

This Actor combines Unicode-aware candidate detection with URL parsing and public-suffix classification. Unknown suffixes and malformed IP addresses are rejected instead of being silently emitted as valid public domains.

Page fetching is deliberately bounded. Every page is validated as a public HTTP(S) target, redirects are checked again, and private/local network addresses are rejected.

### Input options

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `text` | string | empty | Pasted prose, logs, CSV text, email signatures, or other text to scan. Maximum 2,000,000 characters. |
| `urls` | string array | `[]` | URLs or domains to scan without fetching. Maximum 10,000 entries. |
| `startUrls` | request list | `[]` | Public HTTP(S) pages to fetch and scan. Maximum 50 pages. |
| `maxItems` | integer | `20` | Maximum number of unique normalized hostnames returned. Range 1–10,000. |
| `normalizeWww` | boolean | `true` | Remove a leading `www.` before deduplication. |
| `includeIpAddresses` | boolean | `false` | Include valid IPv4 addresses as separately classified rows. |

Provide at least one non-empty `text`, `urls`, or `startUrls` value.

### Getting started

1. Open the Actor input page.
2. Paste text into **Text to scan**, add entries to **URL or domain list**, or add public pages.
3. Keep **Merge www hostnames** enabled for apex-level list cleanup.
4. Set **Maximum unique domains** to bound the dataset size.
5. Run the Actor.
6. Open the default dataset and use the **Domains** view.
7. Export JSON, CSV, Excel, XML, or RSS through Apify dataset exports.

A simple input is:

```json
{
  "text": "Visit https://www.apify.com/store, docs.apify.com and support@crawlee.dev. APIFY.com appears again.",
  "normalizeWww": true,
  "maxItems": 100
}
```

### Extracted domain data

Each unique normalized hostname produces one row.

| Field | Meaning |
| --- | --- |
| `hostname` | Lowercase ASCII/punycode hostname used as the deduplication key. |
| `unicodeHostname` | Human-readable Unicode form of an internationalized hostname. |
| `registrableDomain` | Domain plus public suffix, excluding subdomains; null for IP addresses. |
| `publicSuffix` | ICANN or recognized private suffix such as `com`, `co.uk`, or `github.io`; null for IP addresses. |
| `subdomain` | Portion before the registrable domain, or null. |
| `classification` | `registrable-domain`, `subdomain`, or `ip-address`. |
| `isIpAddress` | Whether the row represents an opted-in IPv4 address. |
| `occurrenceCount` | Number of matching occurrences merged into the row. |
| `sourceTypes` | Input routes where the hostname appeared: text, URL list, or web page. |
| `sources` | Input references or final fetched page URLs containing the hostname. |
| `firstMatchedValue` | First original candidate that produced the row. |
| `extractedAt` | ISO 8601 extraction timestamp. |

### Output example

For the getting-started input, one emitted row is:

```json
{
  "hostname": "apify.com",
  "unicodeHostname": "apify.com",
  "registrableDomain": "apify.com",
  "publicSuffix": "com",
  "subdomain": null,
  "classification": "registrable-domain",
  "isIpAddress": false,
  "occurrenceCount": 2,
  "sourceTypes": ["text"],
  "sources": ["pasted-text"],
  "firstMatchedValue": "https://www.apify.com/store,",
  "extractedAt": "2026-09-04T14:00:00.000Z"
}
```

Rows are written to the run's default dataset, so standard Apify integrations work without a custom dataset name.

### Domain normalization behavior

Hostnames are converted to lowercase ASCII for the `hostname` key.

Internationalized labels are converted to punycode for stable matching and returned in readable form through `unicodeHostname`.

When `normalizeWww` is true, only the leading `www.` label is removed. Other subdomains are preserved because they can represent distinct infrastructure or business functions.

Deduplication is global across all input routes in one run. If a hostname appears in both pasted text and a URL entry, `occurrenceCount`, `sourceTypes`, and `sources` capture that relationship.

### Public page scanning and safety

`startUrls` fetches only the supplied pages. It does not discover or crawl linked pages.

Each request:

- accepts only HTTP and HTTPS;
- rejects embedded URL credentials;
- resolves and rejects private, loopback, link-local, and reserved network targets;
- validates every redirect target;
- follows at most five redirects;
- retries transient timeouts, HTTP 429, and selected 5xx responses at most twice;
- times out after 20 seconds per request;
- accepts text, HTML, and XML-like responses;
- rejects responses larger than 2 MB.

JavaScript-rendered links that are absent from the initial response are not extracted. Paste rendered content into `text` when you already have it.

### How much does it cost to extract unique domains?

Pay-per-event pricing contains a **$0.005 start fee** plus a fee for each unique domain row produced.

At the BRONZE tier, each unique domain currently costs **$0.00064384**. Duplicate occurrences do not create additional event charges, and a run with no results has only the start event.

Examples at BRONZE pricing:

| Unique domains | Estimated event price |
| ---: | ---: |
| 1 | $0.00564384 |
| 25 | $0.021096 |
| 100 | $0.069384 |
| 1,000 | $0.64884 |

Apify selects the account tier and displays the applicable price before a run. Platform compute or proxy treatment follows the pricing shown by Apify; this Actor does not automatically use a residential proxy.

### Workflow ideas

#### Clean a lead list

Pass mixed company URLs through `urls`, merge `www`, and join downstream records by `registrableDomain`.

#### Build a security indicator inventory

Paste an incident report or log excerpt into `text`, enable IPv4 output when needed, and group rows by `classification`.

#### Inspect one public page

Supply its URL in `startUrls` and review `hostname`, `registrableDomain`, and `sourceTypes` for linked infrastructure.

#### Schedule recurring normalization

Create an Apify Task with a stable input, schedule it, and consume the newest dataset from a webhook or automation platform.

### API usage with cURL

Set `APIFY_TOKEN` without committing it to source control.

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~domain-name-extractor/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.apify.com/store","docs.apify.com"],"maxItems":100}'
```

To wait for completion and receive dataset items directly:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~domain-name-extractor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text":"apify.com and docs.apify.com"}'
```

### API usage with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/domain-name-extractor').call({
    urls: ['https://www.apify.com/store', 'docs.apify.com'],
    normalizeWww: true,
    maxItems: 100,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### API usage with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/domain-name-extractor').call(run_input={
    'text': 'Review apify.com, docs.apify.com, and support@crawlee.dev',
    'normalizeWww': True,
    'maxItems': 100,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with Apify MCP

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/domain-name-extractor"
```

Claude Desktop, Cursor, and VS Code can use the equivalent HTTP MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/domain-name-extractor"
    }
  }
}
```

Example prompts:

- “Extract and deduplicate domains from this incident report.”
- “Normalize this company URL list and keep meaningful subdomains.”
- “Scan this public page and return its linked domain inventory.”

### Limits and failure behavior

The Actor emits at most 10,000 unique rows per run.

The text field supports up to 2,000,000 characters, URL lists support 10,000 entries, and page mode supports 50 supplied pages.

Unknown public suffixes are ignored. This avoids treating version strings or arbitrary dotted tokens as public domains.

If one supplied page fails permanently, the run fails rather than presenting a partial dataset as complete. Split unreliable sources into separate tasks if independent completion is more useful.

`maxItems` stops accepting new unique hostnames but still aggregates repeated occurrences already encountered for accepted hostnames.

### Legality and responsible use

Process only data you are authorized to use.

Public availability does not automatically grant every reuse right. Follow applicable website terms, robots guidance, intellectual-property rules, privacy laws, and organizational policies.

Do not use page scanning to probe private infrastructure. The Actor rejects private and local network destinations, but users remain responsible for lawful inputs and downstream handling.

Avoid placing secrets, private customer data, or access tokens in text that does not need to leave your controlled workflow.

### Troubleshooting

**The dataset is empty.** Confirm the input contains valid domains with recognized ICANN or private suffixes. Unknown internal suffixes such as `.local` are intentionally excluded.

**`www` disappeared.** This is expected when `normalizeWww` is true. Disable it if `www.example.com` must remain a separate hostname.

**A page run failed with a private-target error.** Use a publicly resolvable HTTP(S) page. Localhost, intranet, and private IP targets are not supported.

**A JavaScript-only link is missing.** Page mode scans the initial response and does not run a browser. Supply rendered text directly if you are authorized to process it.

**The output contains subdomains separately.** This is intentional. Use `registrableDomain` to group every hostname belonging to the same registrable domain.

### Related Automation Lab Actors

- [Domain Registration & RDAP Lookup](https://apify.com/automation-lab/domain-rdap-lookup) enriches extracted domains with public registration data.
- [SSL Certificate Checker](https://apify.com/automation-lab/ssl-certificate-checker) checks certificate details for known hosts.
- [Sitewide Broken Link Checker](https://apify.com/automation-lab/sitewide-broken-link-checker) crawls a site when you need link health rather than domain inventory.
- [Company Domain and Social Links Finder](https://apify.com/automation-lab/company-domain-social-links-finder) starts from company identity and discovers company web/social links.

### FAQ

#### Does the Actor crawl an entire website?

No. Every `startUrls` entry is fetched once, after bounded redirects and transient retries. Links are scanned but not followed.

#### Does it validate that every domain currently resolves?

No. Public-suffix and syntax validation are deterministic; DNS availability can change and is not part of the output contract.

#### Can it extract domains from email addresses?

Yes. A domain embedded after `@` is detected and normalized, but the email address itself is not returned.

#### Are duplicates charged more than once?

No. The per-domain event is emitted only for unique output rows. `occurrenceCount` records duplicates inside the row.

#### Can it include IP addresses?

Valid IPv4 addresses are optional through `includeIpAddresses`. IPv6 extraction is not currently supported.

#### Can I export to Excel or CSV?

Yes. Use the standard default-dataset export controls or API formats provided by Apify.

# Actor input Schema

## `text` (type: `string`):

Paste prose, logs, email signatures, CSV text, or any other text containing domain names and URLs.

## `urls` (type: `array`):

Paste or provide individual URLs/domains. Each entry is scanned but not fetched.

## `startUrls` (type: `array`):

Optional anonymously reachable HTTP(S) pages. The Actor scans page text and links without crawling beyond each supplied page.

## `maxItems` (type: `integer`):

Stop output after this many unique normalized hostnames.

## `normalizeWww` (type: `boolean`):

Remove a leading www. before deduplication so www.example.com and example.com become one record.

## `includeIpAddresses` (type: `boolean`):

Also return valid IPv4 addresses found in the input, classified separately from domains.

## Actor input object example

```json
{
  "text": "Partners: https://www.apify.com/store and hello@crawlee.dev. Review docs.apify.com and APIFY.com again.",
  "urls": [],
  "startUrls": [],
  "maxItems": 20,
  "normalizeWww": true,
  "includeIpAddresses": false
}
```

# Actor output Schema

## `overview` (type: `string`):

Unique normalized domains, subdomains, and optional IPv4 addresses found across all supplied inputs.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "text": "Partners: https://www.apify.com/store and hello@crawlee.dev. Review docs.apify.com and APIFY.com again."
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/domain-name-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "text": "Partners: https://www.apify.com/store and hello@crawlee.dev. Review docs.apify.com and APIFY.com again." }

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/domain-name-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "text": "Partners: https://www.apify.com/store and hello@crawlee.dev. Review docs.apify.com and APIFY.com again."
}' |
apify call automation-lab/domain-name-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/domain-name-extractor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/XQySQKCOKwdXGL62W/builds/Zxu5o51icmbwVKHkF/openapi.json