# URL Categorizer DomainScope (`domainscope/domainscope-url-categorizer`) Actor

Categorize URLs by topic, industry, business model, target market, and audience profile using the DomainScope API.

- **URL**: https://apify.com/domainscope/domainscope-url-categorizer.md
- **Developed by:** [Florin Badita](https://apify.com/domainscope) (community)
- **Categories:** AI, Developer tools, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 url categorizeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## DomainScope URL Categorizer — Apify Actor

A thin Apify Actor wrapper around the [go-url-categorizer-api](../) service
("DomainScope"), deployed publicly at `https://domainscope.scrapetheworld.org`.
It does not reimplement any categorization logic — it forwards `urls` to the
existing REST API and streams the results into the Actor's dataset.

### How it decides which endpoint to call

| Input                              | Endpoint(s) used                                                      |
|-------------------------------------|-------------------------------------------------------------------------|
| 1 URL                                | `POST /api/v1/categorize` (auth optional)                              |
| 2..`jobThreshold` URLs (default 25)  | `POST /api/v1/categorize/bulk` (auth required)                         |
| more than `jobThreshold` URLs        | `POST /api/v1/jobs/bulk-categorize`, polled via `GET /jobs/{id}/status`, fetched via `GET /jobs/{id}/download` (auth required) |

The async job path uses a random `Idempotency-Key` per run so a retried
submission cannot double-queue (and double-bill) the same URLs.

### Input

See [`.actor/input_schema.json`](.actor/input_schema.json). Key fields:

- **`urls`** (required) — list of URLs/domains to categorize.
- **`apiKey`** — Bearer token for the DomainScope API. Required for more
  than one URL. If left blank, the Actor falls back to the `DOMAINSCOPE_API_KEY`
  environment variable (see "Shared service key" below), then to the
  anonymous tier (single URL only, 10 requests/hour).
- **`apiBaseUrl`** — defaults to `https://domainscope.scrapetheworld.org/api/v1`.
  Override for local/staging testing.
- **`jobThreshold`**, **`pollIntervalSecs`**, **`maxPollMinutes`** — tune the
  sync-vs-async cutover and how long the Actor waits on a large job.

### Output

One dataset row per input URL, shaped like `model.CategoryResult` /
`model.CategoryResultWithError` from the API
([`openapi/openapi.yaml`](../openapi/openapi.yaml)): `url`, `category`,
`sub_category`, `industry`, `country`, `business_model`, `target_market`,
`audience_profile`, `content_focus`, `entity_type`, `city`, `summary`, and
`error` when categorization failed for that URL.

### Billing

The Actor charges one `url-categorized` event (via `Actor.charge()`) per
**successfully** categorized URL — rows with an `error` are pushed to the
dataset but not charged, mirroring the API's own bulk/job semantics ("a
per-URL failure does not fail the request").

`Actor.charge()` is a documented no-op outside a pay-per-event-enabled run
(e.g. `apify run` locally), so local development works without any billing
setup. To publish this as a monetized Actor on Apify Store:

1. Push the Actor to Apify (`apify push`) and open it in the Apify Console.
2. Under **Publication → Monetization**, choose **Pay per event** and add a
   `url-categorized` event with whatever price you want to charge — the
   event name must match the string used in `src/main.js`.
3. Retrieve the actor's chargeable event names via the
   [Get Actor API endpoint](https://docs.apify.com/api/v2) to confirm the
   config took effect before going live.

### Shared service key vs. bring-your-own key

Two ways to run this in production, pick one:

- **BYO key (default, no extra setup)** — each caller supplies their own
  DomainScope API key via the `apiKey` input field. They're billed by
  DomainScope's own credit system; the Actor is unmetered (or metered
  separately via Apify PPE on top, if you want to charge for the
  convenience wrapper itself).
- **Shared service key** — mint one API key for the Actor itself
  (`POST /api/v1/api-keys` against the DomainScope API, see
  [`API_DOCS.md`](../API_DOCS.md)), store it as an Apify Actor secret
  environment variable named `DOMAINSCOPE_API_KEY` (Console → Actor →
  Settings → Environment variables → mark as secret), and leave `apiKey`
  blank in the input. End users never see the key; you bill them entirely
  through Apify's pay-per-event pricing instead of DomainScope credits.

Either way, no changes are needed on the DomainScope API side — it already
has a dedicated API-key rate-limit tier (6000 req/min by default,
`src/internal/middleware/ratelimit.go`) separate from the anonymous and
per-user tiers, so a shared service key comfortably absorbs Actor traffic
without needing IP allowlisting (which wouldn't work for Apify's cloud
workers anyway — they have no fixed egress IP range).

### Local development

```bash
cd apify-actor
npm install
cp .env.example .env   # fill in DOMAINSCOPE_API_KEY for multi-URL testing
export $(grep -v '^#' .env | xargs)

## Provide input via storage/key_value_stores/default/INPUT.json, or:
echo '{"urls": ["https://example.com", "https://anthropic.com"]}' \
  > storage/key_value_stores/default/INPUT.json

npm run start:dev
```

Or with the Apify CLI (`npm i -g apify-cli`):

```bash
apify run --input '{"urls": ["https://example.com"]}'
```

### Deploying

```bash
apify login
apify push
```

This builds `Dockerfile` on Apify's infrastructure and creates/updates the
Actor under your Apify account. See [Apify's Actor deployment docs](https://docs.apify.com/platform/actors/development/deployment)
for CI-based alternatives.

# Actor input Schema

## `urls` (type: `array`):

URLs or bare domains to categorize, e.g. "https://example.com" or "example.com". One dataset row is produced per URL.

## `apiKey` (type: `string`):

Bearer token for the DomainScope API (issued via POST /api/v1/api-keys). Required for lists longer than one URL. Leave blank only for a single-URL run, which falls back to the heavily rate-limited (10/hour) anonymous tier. If the Actor operator has configured a shared key, leave blank to use it.

## `apiBaseUrl` (type: `string`):

Base URL of the DomainScope API, including the /api/v1 path. Change only for staging/testing against a non-production instance.

## `jobThreshold` (type: `integer`):

Lists longer than this many URLs are submitted as an async bulk job (POST /jobs/bulk-categorize) and polled to completion, instead of one synchronous POST /categorize/bulk call, so large runs cannot time out.

## `pollIntervalSecs` (type: `integer`):

How often to poll GET /jobs/{id}/status while an async bulk job is running.

## `maxPollMinutes` (type: `integer`):

Abort and fail the run if the async bulk job has not finished within this many minutes.

## Actor input object example

```json
{
  "urls": [
    "https://example.com"
  ],
  "apiBaseUrl": "https://domainscope.scrapetheworld.org/api/v1",
  "jobThreshold": 25,
  "pollIntervalSecs": 5,
  "maxPollMinutes": 30
}
```

# Actor output Schema

## `categorizedUrls` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("domainscope/domainscope-url-categorizer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://example.com"] }

# Run the Actor and wait for it to finish
run = client.actor("domainscope/domainscope-url-categorizer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com"
  ]
}' |
apify call domainscope/domainscope-url-categorizer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,domainscope/domainscope-url-categorizer"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bImFkdjMG7j514R6w/builds/N2s5Mg7igm97HPTK1/openapi.json