# Website Contact & Company Data Scraper with AI Lead Scoring (`craftedincode/website-company-profile-ai`) Actor

Crawl company domains for emails, phones, and socials, then add an AI company profile and ICP fit score. Flat price per domain, not per page.

- **URL**: https://apify.com/craftedincode/website-company-profile-ai.md
- **Developed by:** [Crafted in Code](https://apify.com/craftedincode) (community)
- **Categories:** Lead generation, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 domain crawleds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Contact & Company Data Scraper with AI Lead Scoring

Paste a list of **company domains**, or pass a dataset from a **Google Maps scraper**, and get a **company profile** for each site: emails, phone numbers, social profiles, a few tech hints, and an optional **AI lead score** against your ideal customer profile.

You pay a **flat price per domain**, not per page. The crawl is limited to the homepage plus contact, about, team, careers, and legal pages (8 pages max). Dead, parked, and blocked domains are returned with a `status` and are **not charged**.

Built for agencies, SDRs, and Clay / n8n / Make workflows that already have a lead list and need qualification without a surprise crawl bill.

### What data can you extract?

| Field | What it is |
|---|---|
| `domain`, `finalUrl`, `companyName`, `title` | The site that was crawled, after redirects, and the name from the page title or structured data |
| `status` | `ok`, `dead`, `parked`, `blocked`, or `error` |
| `emails` | Role inboxes such as `info@` and `sales@` first. Named personal emails are omitted unless you opt in |
| `phones` | Numbers from `tel:` links, structured data, and formatted numbers on the page |
| `socials` | LinkedIn, Facebook, Instagram, X, YouTube, TikTok |
| `techHints` | Signatures such as Shopify, WordPress, HubSpot, GA4, Stripe, Intercom |
| `contactPageUrl`, `addressHint`, `countryHint`, `hasCareersPage` | Contact page, a short address when the site publishes one, careers link |
| `pagesCrawled`, `pageUrls` | How many pages were actually fetched |
| `aiSummary`, `industry`, `subIndustry`, `businessModel` | Short description and category. Industry is one of a fixed list |
| `offerings`, `targetCustomers`, `pricingModelHint` | What they sell, who they sell to, and a pricing hint (`demo`, `quote`, `ecommerce`, …) |
| `companySizeSignal`, `companySizeEvidence` | `solo`, `SMB`, `mid`, `enterprise`, or `unknown`, plus the phrase that suggested it |
| `icpFitScore`, `icpFitReasons`, `personalizationLine` | 0–100 against your ICP text, the reasons, and one outreach opener |
| `aiConfidence`, `aiMock`, `aiError` | How sure the model was, whether the offline mock was used, and why AI was skipped |
| `error`, `chargedEvents`, `source` | Failure message, which events were billed, and original columns from a source dataset |

`icpFitScore` is null when you do not provide an ICP. The model is instructed to use only the crawled text. It can still be wrong. Use `aiConfidence` and `companySizeEvidence` before you trust a score.

### How to use it

1. Open the Actor and paste domains (`acme.com` or `https://acme.com/pricing`) **or** pick a dataset and the column that holds the website (`website` for Google Maps).
2. Set **Max domains**. This is a hard stop.
3. Optionally set **Max charge (USD)**. The run also respects the maximum charge you set on the Apify run itself. The lower limit wins.
4. Leave **AI company profile** on, and describe your ideal customer in plain English. Example: `B2B SaaS, 20–500 employees, sells to HR teams in the US and UK`.
5. Run. Each `ok` domain is one `domain-crawled` event. Each AI profile is one `ai-profile` event. The status message explains when a cap stops the run.

The same domain is crawled once, even if you paste both `www.acme.com` and `https://acme.com/about`. Deep links are fetched in addition to the homepage, still inside the page cap.

Free plan: at most **25 AI profiles per run** and **100 per UTC month**. The crawl can continue without AI after that, until the domain or charge cap. The row's `aiError` says which cap applied.

### How much does it cost to scrape company websites?

Pricing is pay-per-event. You are not billed per page, and you are not billed for a domain that did not resolve, looks parked, or blocked the crawler.

| Event | Free plan | Paid plans (Bronze and up) | What you pay for |
|---|---|---|---|
| `domain-crawled` | $0.004 | $0.003 | One domain with a usable page |
| `ai-profile` | $0.030 | $0.010 | One AI company profile |
| Start event | platform default | platform default | A small platform charge when the run starts |

Paid-plan examples:

| Run | Price |
|---|---|
| 100 domains, contacts only | $0.30 |
| 100 domains with AI profiles | $1.30 |
| 1,000 domains, contacts only | $3.00 |
| 1,000 domains with AI profiles | $13.00 |
| 1 domain, the prefilled test | $0.003 + $0.01 if AI runs |

Free-plan example: 25 domains with AI is `25 × $0.004 + 25 × $0.03 = $0.85`, and further AI in that UTC month is skipped after 100 profiles.

Set a max charge on the run if you want a hard dollar stop. `maxDomains` stops the crawl even when money remains. Rows already in flight in the current small batch are finished; new domains are not started.

#### Flat price vs per-page contact scrapers

| | This Actor | Typical per-page contact scraper |
|---|---|---|
| Unit | One domain, up to 8 pages | Every page fetched |
| A 5-page site | $0.003 on paid plans | Often $0.002 × 5 pages, plus browser or proxy add-ons |
| A site that explodes into hundreds of URLs | Stopped by the page cap | Can become a large bill and a huge dataset |
| Dead or parked domain | Output row, **$0** | Often still billed per request |
| Lead qualification | Optional AI profile and ICP score | Usually not included |
| Google Maps dataset | Pass the dataset id, keep original columns | Usually a separate copy-paste step |

The $13 / 1,000 fully enriched domains figure is the paid-plan list price above. It is not a per-page estimate.

### Input example

```json
{
  "domains": ["acme.com", "https://example.co.uk/about"],
  "datasetId": "yourGoogleMapsDatasetId",
  "websiteField": "website",
  "maxDomains": 100,
  "maxPagesPerDomain": 5,
  "maxChargeUsd": 5,
  "aiEnrichment": true,
  "icpDescription": "B2B SaaS, 20-500 employees, sells to HR teams in the US and UK",
  "aiLanguage": "en",
  "maxAiDomains": 100,
  "includePersonalEmails": false
}
```

The saved default input is a single fast site (`https://crawlee.dev`) so the daily Store check finishes in under five minutes.

### Output example

One dataset item per domain. A full local sample, produced with the offline mock model against live sites, is in [`examples/sample-output.json`](examples/sample-output.json). Real runs set `aiMock` to `false` and `aiProvider` to `gemini` or `openai`.

```json
{
  "domain": "acme.com",
  "finalUrl": "https://acme.com/",
  "status": "ok",
  "companyName": "Acme Payroll",
  "emails": ["sales@acme.com"],
  "personalEmailsOmitted": 1,
  "phones": ["+1-415-555-0134"],
  "socials": { "linkedin": "https://www.linkedin.com/company/acme", "x": null, "facebook": null, "instagram": null, "youtube": null, "tiktok": null },
  "techHints": ["HubSpot", "Google Analytics 4"],
  "hasCareersPage": true,
  "pagesCrawled": 3,
  "aiSummary": "Acme sells payroll software to HR teams.",
  "industry": "Software",
  "businessModel": "B2B",
  "companySizeSignal": "SMB",
  "icpFitScore": 78,
  "icpFitReasons": ["Sells to HR teams", "B2B software"],
  "personalizationLine": "Noticed Acme is focused on HR payroll.",
  "aiConfidence": 0.7,
  "aiMock": false,
  "chargedEvents": ["domain-crawled", "ai-profile"],
  "error": null,
  "aiError": null
}
```

A blocked site looks like `{ "status": "blocked", "error": "The site blocked the crawler or returned a bot challenge.", "chargedEvents": [] }` and does not add an AI charge.

Run totals, including an estimated LLM cost versus AI-event revenue, are stored as the key-value record `RUN_STATS`.

### FAQ

**Is scraping public company websites legal?**
This Actor fetches pages a browser could open and extracts business contact details the site publishes. It does not log in, bypass paywalls, or solve CAPTCHAs. Laws differ by country. You are responsible for having a lawful basis, especially under GDPR, and for following CAN-SPAM, CASL, and similar rules if you email anyone. This is not legal advice.

**Why are some emails missing?**
By default only role-based addresses are returned (`info@`, `sales@`, `hello@`, and similar). A named inbox such as `jane.doe@` is personal data and is counted in `personalEmailsOmitted` instead. Turn on **Include personal emails** only when you have a reason to process that data. Addresses that are clearly placeholders, or that belong to vendors such as Sentry, are dropped.

**Why is there no phone number?**
Numbers are taken from `tel:` links, structured data, and formatted phone patterns. A number that is only drawn as an image, or only rendered by JavaScript, will not be found. This Actor uses HTTP and Cheerio, not a browser, so a site that ships an empty HTML shell may come back thin or `blocked`.

**What does the ICP score mean?**
It is a 0–100 judgment of the crawled text against the ICP description you typed. It is not a credit score and it is not headcount. If the pages are thin, `aiConfidence` is low. If you leave the ICP blank, the score is null.

**What happens at the free-plan limit?**
The run does not crash. Further AI calls are skipped, `aiError` explains the cap, and contact rows that were already allowed are still saved. Raising `maxAiDomains` does not override the free-plan cap.

**Can I connect it to Make, n8n, Zapier, or an AI agent?**
Yes. Run it from the Apify API, a saved task, or an integration. Map the dataset id of a Google Maps actor into `datasetId`. The output dataset is a normal Apify dataset, so Make, n8n, Zapier, and the Apify MCP can read the items. The Actor uses limited permissions and pay-per-event pricing, which is what agent checkout flows expect.

**Does it charge when the model fails?**
No. `ai-profile` is charged only after the response parses as a profile. A timeout or invalid JSON sets `aiError` and does not bill that event. The crawl event is also skipped unless `status` is `ok`.

### Integrations

- **Apify API:** start a run with the input JSON above and read the default dataset.
- **Google Maps scrapers:** pass their dataset id. The website column is picked up automatically.
- **Make, n8n, Zapier:** use the Apify integration and map `emails`, `icpFitScore`, and `personalizationLine`.
- **Clay:** call the run-sync endpoint and spread the dataset items into rows.
- **MCP / agents:** limited permissions, no Standby mode, pay-per-event only.

### Limits and honest gaps

- JavaScript-only sites may return little text. There is no Playwright fallback, on purpose: a browser would make the flat domain price unpredictable.
- The crawler is polite (robots.txt, a short delay, low concurrency) and will record `blocked` instead of hammering a site.
- Industry and company size are model judgments. `unknown` is a valid size signal.
- Email and phone coverage will miss obfuscated addresses (`info [at] acme.com` written as an image, or assembled by JavaScript).

### Local development

```bash
npm install
npm test
npm run e2e
```

`npm run e2e` crawls a few public sites with the mock model and refreshes `examples/sample-output.json`. Set `GEMINI_API_KEY` (and optionally `OPENAI_API_KEY`) to use a real model. See [PUBLISHING.md](PUBLISHING.md) for Store setup, secret env vars, and the exact pay-per-event prices.

# Actor input Schema

## `domains` (type: `array`):

Company domains or page URLs, one per line. https:// is optional. Duplicates of the same domain are crawled once.

## `datasetId` (type: `string`):

Optional Apify dataset, for example the output of a Google Maps scraper. The Actor reads the website field and keeps the original columns on each row.

## `websiteField` (type: `string`):

Column that holds the company website. Also checks website, websiteUrl, domain, url, and web. Google Maps links are ignored.

## `maxDomains` (type: `integer`):

Hard stop. The Actor will not crawl more domains than this, even if the charge limit would allow it.

## `maxPagesPerDomain` (type: `integer`):

Homepage plus contact, about, team, careers, and legal pages, up to this many fetches. Capped at 8 so the flat domain price stays predictable.

## `maxChargeUsd` (type: `number`):

Optional extra ceiling for this run, in USD. It is applied together with the run's maximum charge in the Console (the lower one wins). Leave empty to use only the Console limit. Dead, parked, and blocked domains are not charged.

## `maxConcurrency` (type: `integer`):

How many domains to crawl at once. Lower is more polite.

## `includePersonalEmails` (type: `boolean`):

Off by default. Role inboxes such as info@ and sales@ are always kept. Named addresses such as jane.doe@ are personal data and are omitted unless you turn this on. You are responsible for lawful use.

## `proxyConfiguration` (type: `object`):

Optional Apify proxy. Datacenter is enough for most sites. The RESIDENTIAL group is ignored here because residential traffic is billed to the Actor developer; a capped fallback can be enabled by the developer as a secret.

## `aiEnrichment` (type: `boolean`):

Add a summary, industry, what they sell, a size signal, and an ICP fit score. This is a separate charge from the crawl. Free-plan runs are limited to 25 AI profiles per run and 100 per UTC month.

## `icpDescription` (type: `string`):

Plain English description of who you sell to. The fit score is judged against this text. Leave empty and the score is left blank.

## `aiLanguage` (type: `string`):

Language for the summary and outreach line. Use a short code such as en or de.

## `maxAiDomains` (type: `integer`):

Upper bound on AI profiles this run may create. Free-plan runs are still capped at 25 per run and 100 per month.

## Actor input object example

```json
{
  "domains": [
    "https://crawlee.dev"
  ],
  "websiteField": "website",
  "maxDomains": 100,
  "maxPagesPerDomain": 3,
  "maxConcurrency": 5,
  "includePersonalEmails": false,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "aiEnrichment": true,
  "icpDescription": "Developer tools and web scraping platforms used by software teams",
  "aiLanguage": "en",
  "maxAiDomains": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

Company profiles and contact rows.

## `stats` (type: `string`):

Counts, the stop reason, and the LLM cost meter.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "https://crawlee.dev"
    ],
    "websiteField": "website",
    "maxDomains": 100,
    "maxPagesPerDomain": 3,
    "proxyConfiguration": {
        "useApifyProxy": false
    },
    "aiEnrichment": true,
    "icpDescription": "Developer tools and web scraping platforms used by software teams",
    "aiLanguage": "en"
};

// Run the Actor and wait for it to finish
const run = await client.actor("craftedincode/website-company-profile-ai").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["https://crawlee.dev"],
    "websiteField": "website",
    "maxDomains": 100,
    "maxPagesPerDomain": 3,
    "proxyConfiguration": { "useApifyProxy": False },
    "aiEnrichment": True,
    "icpDescription": "Developer tools and web scraping platforms used by software teams",
    "aiLanguage": "en",
}

# Run the Actor and wait for it to finish
run = client.actor("craftedincode/website-company-profile-ai").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "https://crawlee.dev"
  ],
  "websiteField": "website",
  "maxDomains": 100,
  "maxPagesPerDomain": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "aiEnrichment": true,
  "icpDescription": "Developer tools and web scraping platforms used by software teams",
  "aiLanguage": "en"
}' |
apify call craftedincode/website-company-profile-ai --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,craftedincode/website-company-profile-ai"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/HBxmNbnckUGhrcrI3/builds/QEA3oKPpDPqteX3oK/openapi.json