# Domain List Cleaner — Normalize & Dedupe URLs (`night111/domain-list-cleaner`) Actor

Clean messy domain and URL lists: normalize, strip paths, remove social-only hosts, dedupe. Perfect prep step before contact enrichment.

- **URL**: https://apify.com/night111/domain-list-cleaner.md
- **Developed by:** [ng. night](https://apify.com/night111) (community)
- **Categories:** Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.00 / 1,000 clean domains

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Domain List Cleaner — Normalize & Dedupe URLs

Turn messy domain and URL lists into a **clean, deduplicated set of company websites** — ready for contact enrichment, CRM import, or crawlers.

Paste raw exports from Google Maps, directories, spreadsheets, or CRM → get one clean domain per company.

### What is Domain List Cleaner?

Domain List Cleaner is a lightweight **lead-gen prep Actor** on Apify Store. It does **not** scrape the web. It only cleans your list:

1. Accepts domains, full URLs, or mixed spreadsheet junk
2. Normalizes hosts (`HTTPS`, `www`, paths, query strings)
3. Optionally collapses subdomains to the main site
4. Drops social/platform hosts and free-email domains
5. Dedupes → **one row per domain**

Pair it with **[Website Contact Finder](https://apify.com/night111/website-contact-finder)** (or any contact scraper) for the full enrichment pipeline.

### What does it fix?

| Before | After |
| --- | --- |
| `https://www.Example.com/about?ref=1` | `example.com` |
| `EXAMPLE.COM` | `example.com` |
| `blog.shop.acme.co.uk/post` | `acme.co.uk` (if subdomains off) |
| `facebook.com/brand` | **dropped** (social filter) |
| `john@gmail.com` | **dropped** (free email host) |
| duplicates of the same site | **one row** |

### Why use this Actor?

- 🧹 **Clean inputs = better enrichment** — scrapers waste money on Facebook URLs and junk
- ⚡ **Instant & cheap** — pure CPU, no browser, no proxy needed
- 🔗 **Pipeline-ready** — output `https://domain` URLs you can feed straight into a contact scraper
- 📦 **One domain per row** — easy CSV / API handoff
- 🛡️ **Filters noise** — social hosts, free email providers, IPs, invalid strings

### How to use

1. [Create a free Apify account](https://console.apify.com/sign-up)
2. Open **Domain List Cleaner**
3. Paste domains into the list **or** the raw text box
4. Click **Start**
5. Download the **kept** domains as CSV/JSON, or pipe them into Website Contact Finder

#### Recommended pipeline (Google Maps → contacts)

```
1) Google Maps / Places scraper
      → place websites (often messy: facebook, paths, dupes)
2) Domain List Cleaner   ← you are here
      → one clean https://company.com per business
3) Website Contact Finder
      → emails, phones, socials
```

**One-click alternative:** use **[Maps to Contacts](https://apify.com/night111/maps-to-contacts)** to clean + extract contacts in a single run.

##### Step-by-step with separate Actors

1. Run any [Google Maps scraper](https://apify.com/compass/crawler-google-places) (or directory export).
2. Copy the **website** column (or export CSV).
3. Paste into **Domain List Cleaner** → Start.
4. Take the cleaned `url` column → feed **[Website Contact Finder](https://apify.com/night111/website-contact-finder)** `startUrls`.
5. Download contacts as CSV for CRM / outreach.

### Input

| Field | Description |
| --- | --- |
| **Domains / URLs** | Array of strings or objects with `url` / `domain` / `website` |
| **Raw text** | Optional paste block (newlines, commas, spaces) |
| **Keep subdomains** | Default off → `blog.example.com` becomes `example.com` |
| **Strip www.** | Default on |
| **Drop social hosts** | Default on (Facebook, LinkedIn, X, YouTube, …) |
| **Drop free email hosts** | Default on (gmail, outlook, …) |
| **Output as https URL** | Default on |
| **Also list rejected** | Default off — turn on to debug filtered rows |

#### Example input

```json
{
  "domains": [
    "https://www.Example.com/about?ref=1",
    "EXAMPLE.COM",
    "facebook.com/somepage",
    "https://shop.acme.co.uk/products"
  ],
  "keepSubdomains": false,
  "stripWww": true,
  "dropSocialHosts": true,
  "toHttpsUrl": true
}
```

### Output

Each **kept** domain becomes one dataset item:

```json
{
  "domain": "example.com",
  "url": "https://example.com",
  "original": "https://www.Example.com/about?ref=1",
  "status": "kept",
  "reason": "ok",
  "cleanedAt": "2026-07-28T12:00:00.000Z"
}
```

A summary is also saved to the key-value store as `OUTPUT` (counts + reject reasons).

### Who is this for?

- Sales / SDR teams cleaning exports before enrichment
- Agencies standardizing client lead lists
- Anyone chaining Maps → websites → contact scrapers

### How much does it cost?

**Pay-per-event** pricing:

| Event | Price |
| --- | --- |
| **Clean domain** (one kept dataset row) | **$0.001** → **$1.00 / 1,000** |
| Actor start | **$0** (included) |
| Platform usage | Included for users |

Example: clean **1,000** messy URLs → about **$1** if all are kept (rejected rows are never charged).

Always check the **Pricing** tab for live rates on your plan.

### FAQ

#### Does this scrape websites?

No. It only normalizes and filters the list you provide. No HTTP requests to target sites.

#### Can I feed Google Maps results?

Yes. Map your website field into `domains`, or paste a column of URLs into **raw text**.

#### How do I connect to Website Contact Finder?

Export cleaned `url` values, or use Apify integrations / API to pass the dataset into [Website Contact Finder](https://apify.com/night111/website-contact-finder).

#### Will it keep country domains like `.co.uk`?

Yes — multi-part suffixes such as `co.uk`, `com.au`, `com.hk` are handled with best-effort registrable-domain logic.

### Limitations

- Public-suffix list is a practical subset (not the full Mozilla PSL)
- Internationalized domains (punycode) work best when already ASCII/`xn--`
- Does not verify that a domain resolves or has a website

### Feedback & support

Open an issue on the Actor **Issues** tab. Related Actor: **Website Contact Finder** for emails, phones, and socials.

***

**Keywords:** domain cleaner, URL normalizer, dedupe domains, lead list cleaner, website list cleaner, domain deduplication, prep for email scraper, CRM domain cleanup

# Actor input Schema

## `domains` (type: `array`):

Messy list of domains or full URLs. Each item can be a plain string, or an object with a `url` / `domain` field (compatible with Maps / scraper outputs).

## `domainsText` (type: `string`):

Alternative input: paste a block of text with domains/URLs separated by newlines, commas, or spaces. Merged with the Domains list above.

## `keepSubdomains` (type: `boolean`):

If true, keep blog.example.com as-is. If false, collapse to registrable-style host (example.com / acme.co.uk best-effort).

## `stripWww` (type: `boolean`):

Remove leading www. from hostnames.

## `dropSocialHosts` (type: `boolean`):

Remove facebook, linkedin, instagram, x/twitter, youtube, tiktok, etc. so only real company websites remain.

## `dropEmailHosts` (type: `boolean`):

Remove gmail.com, yahoo.com, outlook.com, and similar free-mail hosts if they appear as 'domains'.

## `toHttpsUrl` (type: `boolean`):

If true, each result includes a ready-to-use startUrl for crawlers (https://domain).

## `includeRejected` (type: `boolean`):

Push rejected/invalid/social-filtered items to the dataset with status=rejected (useful for debugging).

## Actor input object example

```json
{
  "domains": [
    "https://www.Example.com/about?ref=1",
    "EXAMPLE.COM",
    "http://blog.example.com/post",
    "facebook.com/somepage",
    "not a domain!!!",
    "https://shop.acme.co.uk/products"
  ],
  "keepSubdomains": false,
  "stripWww": true,
  "dropSocialHosts": true,
  "dropEmailHosts": true,
  "toHttpsUrl": true,
  "includeRejected": false
}
```

# Actor output Schema

## `cleanedDomains` (type: `string`):

Dataset of kept domains (and optional rejected rows if enabled).

## `summary` (type: `string`):

Counts, sample kept domains, reject reasons.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "https://www.Example.com/about?ref=1",
        "EXAMPLE.COM",
        "http://blog.example.com/post",
        "facebook.com/somepage",
        "not a domain!!!",
        "https://shop.acme.co.uk/products"
    ],
    "domainsText": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("night111/domain-list-cleaner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": [
        "https://www.Example.com/about?ref=1",
        "EXAMPLE.COM",
        "http://blog.example.com/post",
        "facebook.com/somepage",
        "not a domain!!!",
        "https://shop.acme.co.uk/products",
    ],
    "domainsText": "",
}

# Run the Actor and wait for it to finish
run = client.actor("night111/domain-list-cleaner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "https://www.Example.com/about?ref=1",
    "EXAMPLE.COM",
    "http://blog.example.com/post",
    "facebook.com/somepage",
    "not a domain!!!",
    "https://shop.acme.co.uk/products"
  ],
  "domainsText": ""
}' |
apify call night111/domain-list-cleaner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=night111/domain-list-cleaner",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/bLFDHeUU95vixfJLy/builds/8z0yeOEi4GQf0JkzQ/openapi.json