Extract Domains From Text

Extract domains from text, URL lists, and bounded public web pages, then normalize and deduplicate the results into analysis-ready rows.

The Actor turns messy lead lists, logs, prose, email addresses, and page links into a consistent domain inventory. Each result includes the normalized hostname, registrable domain, public suffix, subdomain classification, occurrence count, and source lineage.

It runs entirely without a browser or login. Text and URL-list inputs are processed locally; optional page inputs use bounded anonymous HTTP requests.

What this domain name extractor does

Use the Actor to:

• find domains inside pasted prose, logs, CSV text, and email addresses;
• normalize uppercase and internationalized domain names;
• merge www hostnames with their apex hostname when requested;
• keep meaningful subdomains such as docs.apify.com separate;
• identify the registrable domain and public suffix;
• deduplicate repeated occurrences while preserving their count;
• show which input route and source contained each hostname;
• optionally include and classify valid IPv4 addresses;
• scan the initial HTML or text response of supplied public pages.

The Actor does not perform DNS, WHOIS, or RDAP enrichment and does not crawl beyond each supplied page.

Who is it for

Lead operations teams can clean mixed website columns before enrichment or outreach.

Security analysts can turn incident notes, logs, and threat reports into a unique hostname inventory.

SEO and link analysts can list domains and subdomains present in a page response.

Data engineers can normalize domain keys before joining datasets or sending records to another Actor.

Developers can replace one-off domain regex scripts with a reusable API endpoint and scheduled task.

Why use this Actor

A regular expression can find domain-like strings, but it cannot reliably classify public suffixes such as co.uk, normalize IDNs, merge duplicates, or retain source lineage.

This Actor combines Unicode-aware candidate detection with URL parsing and public-suffix classification. Unknown suffixes and malformed IP addresses are rejected instead of being silently emitted as valid public domains.

Page fetching is deliberately bounded. Every page is validated as a public HTTP(S) target, redirects are checked again, and private/local network addresses are rejected.

Input options

Field	Type	Default	Description
text	string	empty	Pasted prose, logs, CSV text, email signatures, or other text to scan. Maximum 2,000,000 characters.
urls	string array	[]	URLs or domains to scan without fetching. Maximum 10,000 entries.
startUrls	request list	[]	Public HTTP(S) pages to fetch and scan. Maximum 50 pages.
maxItems	integer	20	Maximum number of unique normalized hostnames returned. Range 1–10,000.
normalizeWww	boolean	true	Remove a leading www. before deduplication.
includeIpAddresses	boolean	false	Include valid IPv4 addresses as separately classified rows.

Provide at least one non-empty text, urls, or startUrls value.

Getting started

1. Open the Actor input page.
2. Paste text into Text to scan, add entries to URL or domain list, or add public pages.
3. Keep Merge www hostnames enabled for apex-level list cleanup.
4. Set Maximum unique domains to bound the dataset size.
5. Run the Actor.
6. Open the default dataset and use the Domains view.
7. Export JSON, CSV, Excel, XML, or RSS through Apify dataset exports.

A simple input is:

{
  "text": "Visit https://www.apify.com/store, docs.apify.com and support@crawlee.dev. APIFY.com appears again.",
  "normalizeWww": true,
  "maxItems": 100
}

Extracted domain data

Each unique normalized hostname produces one row.

Field	Meaning
hostname	Lowercase ASCII/punycode hostname used as the deduplication key.
unicodeHostname	Human-readable Unicode form of an internationalized hostname.
registrableDomain	Domain plus public suffix, excluding subdomains; null for IP addresses.
publicSuffix	ICANN or recognized private suffix such as com, co.uk, or github.io; null for IP addresses.
subdomain	Portion before the registrable domain, or null.
classification	registrable-domain, subdomain, or ip-address.
isIpAddress	Whether the row represents an opted-in IPv4 address.
occurrenceCount	Number of matching occurrences merged into the row.
sourceTypes	Input routes where the hostname appeared: text, URL list, or web page.
sources	Input references or final fetched page URLs containing the hostname.
firstMatchedValue	First original candidate that produced the row.
extractedAt	ISO 8601 extraction timestamp.

Output example

For the getting-started input, one emitted row is:

{
  "hostname": "apify.com",
  "unicodeHostname": "apify.com",
  "registrableDomain": "apify.com",
  "publicSuffix": "com",
  "subdomain": null,
  "classification": "registrable-domain",
  "isIpAddress": false,
  "occurrenceCount": 2,
  "sourceTypes": ["text"],
  "sources": ["pasted-text"],
  "firstMatchedValue": "https://www.apify.com/store,",
  "extractedAt": "2026-09-04T14:00:00.000Z"
}

Rows are written to the run's default dataset, so standard Apify integrations work without a custom dataset name.

Domain normalization behavior

Hostnames are converted to lowercase ASCII for the hostname key.

Internationalized labels are converted to punycode for stable matching and returned in readable form through unicodeHostname.

When normalizeWww is true, only the leading www. label is removed. Other subdomains are preserved because they can represent distinct infrastructure or business functions.

Deduplication is global across all input routes in one run. If a hostname appears in both pasted text and a URL entry, occurrenceCount, sourceTypes, and sources capture that relationship.

Public page scanning and safety

startUrls fetches only the supplied pages. It does not discover or crawl linked pages.

Each request:

• accepts only HTTP and HTTPS;
• rejects embedded URL credentials;
• resolves and rejects private, loopback, link-local, and reserved network targets;
• validates every redirect target;
• follows at most five redirects;
• retries transient timeouts, HTTP 429, and selected 5xx responses at most twice;
• times out after 20 seconds per request;
• accepts text, HTML, and XML-like responses;
• rejects responses larger than 2 MB.

JavaScript-rendered links that are absent from the initial response are not extracted. Paste rendered content into text when you already have it.

How much does it cost to extract unique domains?

Pay-per-event pricing contains a $0.005 start fee plus a fee for each unique domain row produced.

At the BRONZE tier, each unique domain currently costs $0.00064384. Duplicate occurrences do not create additional event charges, and a run with no results has only the start event.

Examples at BRONZE pricing:

Unique domains	Estimated event price
1	$0.00564384
25	$0.021096
100	$0.069384
1,000	$0.64884

Apify selects the account tier and displays the applicable price before a run. Platform compute or proxy treatment follows the pricing shown by Apify; this Actor does not automatically use a residential proxy.

Workflow ideas

Clean a lead list

Pass mixed company URLs through urls, merge www, and join downstream records by registrableDomain.

Build a security indicator inventory

Paste an incident report or log excerpt into text, enable IPv4 output when needed, and group rows by classification.

Inspect one public page

Supply its URL in startUrls and review hostname, registrableDomain, and sourceTypes for linked infrastructure.

Schedule recurring normalization

Create an Apify Task with a stable input, schedule it, and consume the newest dataset from a webhook or automation platform.

API usage with cURL

Set APIFY_TOKEN without committing it to source control.

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~domain-name-extractor/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.apify.com/store","docs.apify.com"],"maxItems":100}'

To wait for completion and receive dataset items directly:

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~domain-name-extractor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"text":"apify.com and docs.apify.com"}'

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/domain-name-extractor').call({
    urls: ['https://www.apify.com/store', 'docs.apify.com'],
    normalizeWww: true,
    maxItems: 100,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/domain-name-extractor').call(run_input={
    'text': 'Review apify.com, docs.apify.com, and support@crawlee.dev',
    'normalizeWww': True,
    'maxItems': 100,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use with Apify MCP

Add the Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/domain-name-extractor"

Claude Desktop, Cursor, and VS Code can use the equivalent HTTP MCP configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/domain-name-extractor"
    }
  }
}

Example prompts:

• “Extract and deduplicate domains from this incident report.”
• “Normalize this company URL list and keep meaningful subdomains.”
• “Scan this public page and return its linked domain inventory.”

Limits and failure behavior

The Actor emits at most 10,000 unique rows per run.

The text field supports up to 2,000,000 characters, URL lists support 10,000 entries, and page mode supports 50 supplied pages.

Unknown public suffixes are ignored. This avoids treating version strings or arbitrary dotted tokens as public domains.

If one supplied page fails permanently, the run fails rather than presenting a partial dataset as complete. Split unreliable sources into separate tasks if independent completion is more useful.

maxItems stops accepting new unique hostnames but still aggregates repeated occurrences already encountered for accepted hostnames.

Legality and responsible use

Process only data you are authorized to use.

Public availability does not automatically grant every reuse right. Follow applicable website terms, robots guidance, intellectual-property rules, privacy laws, and organizational policies.

Do not use page scanning to probe private infrastructure. The Actor rejects private and local network destinations, but users remain responsible for lawful inputs and downstream handling.

Avoid placing secrets, private customer data, or access tokens in text that does not need to leave your controlled workflow.

Troubleshooting

The dataset is empty. Confirm the input contains valid domains with recognized ICANN or private suffixes. Unknown internal suffixes such as .local are intentionally excluded.

www disappeared. This is expected when normalizeWww is true. Disable it if www.example.com must remain a separate hostname.

A page run failed with a private-target error. Use a publicly resolvable HTTP(S) page. Localhost, intranet, and private IP targets are not supported.

A JavaScript-only link is missing. Page mode scans the initial response and does not run a browser. Supply rendered text directly if you are authorized to process it.

The output contains subdomains separately. This is intentional. Use registrableDomain to group every hostname belonging to the same registrable domain.

Related Automation Lab Actors

• Domain Registration & RDAP Lookup enriches extracted domains with public registration data.
• SSL Certificate Checker checks certificate details for known hosts.
• Sitewide Broken Link Checker crawls a site when you need link health rather than domain inventory.
• Company Domain and Social Links Finder starts from company identity and discovers company web/social links.

FAQ

Does the Actor crawl an entire website?

No. Every startUrls entry is fetched once, after bounded redirects and transient retries. Links are scanned but not followed.

Does it validate that every domain currently resolves?

No. Public-suffix and syntax validation are deterministic; DNS availability can change and is not part of the output contract.

Can it extract domains from email addresses?

Yes. A domain embedded after @ is detected and normalized, but the email address itself is not returned.

Are duplicates charged more than once?

No. The per-domain event is emitted only for unique output rows. occurrenceCount records duplicates inside the row.

Can it include IP addresses?

Valid IPv4 addresses are optional through includeIpAddresses. IPv6 extraction is not currently supported.

Can I export to Excel or CSV?

Yes. Use the standard default-dataset export controls or API formats provided by Apify.