# Tor Hidden Service Crawler & Contact Extractor (`xyrelith-data/tor-hidden-service-crawler-and-contact-extractor`) Actor

Crawl Tor .onion sites by URL or keyword search and pull emails, phone numbers, crypto wallet addresses, social handles, and exposed API keys from each page.

- **URL**: https://apify.com/xyrelith-data/tor-hidden-service-crawler-and-contact-extractor.md
- **Developed by:** [Xyrelith Data](https://apify.com/xyrelith-data) (community)
- **Categories:** Lead generation, Automation, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.49 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Onion Directory & Contact Metadata Scraper

Collect public onion directory and contact metadata without crawling page bodies. The actor merges OnionTree, dark.fail and tor.taxi entries by v3 onion domain, keeps provenance, adds neutral categories, descriptions and PGP contact fields, and can measure liveness through a bundled Tor daemon. A separate search mode returns Ahmia index-result metadata only.

### Scope and acceptable use

This actor is **metadata-only by design**. It reads public directory pages, OnionTree YAML records and Ahmia index results. It does **not** collect page text or HTML, credentials, API keys, passwords, cryptocurrency wallet addresses, marketplace listings, vendor/product data, breach data or illegal material. Search-mode result pages are never opened. Liveness checks only fetch the root of a directory entry's onion site to record status and title.

Use it for lawful security research, OSINT triage, directory quality checks and threat-intelligence enrichment. Tor use and onward use of the data may be regulated in your jurisdiction; you are responsible for complying with applicable law and with each source's terms. Do not use it to facilitate crime, to target individuals, or to build a marketplace index. The marketplace name denylist is a best-effort safety filter, not a guarantee.

### Modes

| Mode | What it does | Opens onion pages? |
|---|---|---|
| `directories` (default) | Aggregates OnionTree, dark.fail and tor.taxi, dedupes by v3 onion domain and merges provenance. | Only when `verifyLiveness` is true, one root GET per domain. |
| `search` | Queries the Ahmia index and returns title, URL, rank, snippet and last-seen claim. | Never. |
| `byUrls` | Reads directory/index pages you supply and extracts their v3 onion entries plus adjacent structured contact links. | Fetches the directory pages themselves, never the linked onion entries. |

Legacy mode values are mapped in code: `crawl` becomes `byUrls`, `searchAndCrawl` becomes `directories`, and an unknown mode falls back to `directories`.

### Output fields

Every record is omit-empty: a field the source genuinely lacks is absent, never `null` or an empty array.

**Directory rows (`recordType: "directoryEntry"`)**

- `name`, `title` — entry name (`title` is the legacy alias)
- `onionUrl`, `url`, `onionDomain` — primary URL and its v3 host (`url` is the legacy alias)
- `urls[]` — all v3 mirror URLs for the entry (up to 10)
- `links[]`, `onionLinks[]` — legacy aliases of `urls[]`
- `description` — directory description, capped at 1,000 characters
- `categories[]` — neutral OnionTree categories such as `news`, `mail`, `securedrop`
- `source`, `sourceUrl` — primary provenance (`onionTree`, `darkFail`, `torTaxi`, `byUrls`)
- `alsoListedIn[]` — other sources that listed the same onion domain
- `sourceStatus` — dark.fail's own claim (`online`, `offline`, `unknown`), never used as measured status
- `status`, `httpStatus`, `pageTitle`, `lastCheckedAt` — measured through Tor when `verifyLiveness` is true
- `pgpFingerprint`, `pgpKeyId`, `contactEmail` — OnionTree public-key contact metadata
- `emails[]`, `phones[]`, `misc` — structured contact values only (`mailto:`, `tel:`, explicit social profile links)
- `scrapedAt` — ISO-8601 UTC timestamp

**Search rows (`recordType: "searchResult"`)**

`query`, `rank`, `title`, `onionUrl`, `onionDomain`, `snippet`, `lastSeenClaim`, `totalResults`, `searchKeywordFound`, `source`, `sourceUrl`, `scrapedAt`.

There is no `cryptoAddresses` field and no `misc.apiKeys` field. Both were removed for scope compliance and are never populated.

### Input

| Field | Type | Default | What it controls |
|---|---|---|---|
| `mode` | select | `directories` | `directories`, `search` or `byUrls` |
| `source` | select | `onionTree` | `all`, `onionTree`, `darkFail` or `torTaxi` (directories mode) |
| `categories` | string list | `[]` | OnionTree category filter; empty means all neutral categories |
| `searchQueries` | string list | `[]` | Ahmia queries (search mode; max 20) |
| `maxResultsPerQuery` | integer 1–1000 | `20` | Per-query Ahmia cap |
| `startUrls` | URL list | `[]` | Directory/index pages (byUrls mode) |
| `maxItems` | integer 1–5000 | `100` | Global emitted-record cap |
| `maxPages` | integer 1–50 | `5` | Directory-page cap (byUrls mode) |
| `verifyLiveness` | boolean | `true` | Measure each unique onion domain through Tor |
| `livenessTimeoutSecs` | integer 5–60 | `12` | Per-domain liveness timeout |
| `includeUnreachable` | boolean | `true` | Keep entries measured as `timeout`/`unreachable` |
| `extractEmails` | boolean | `true` | Structured emails from PGP user IDs and `mailto:` links |
| `extractPhones` | boolean | `true` | Phones from explicit `tel:` links only |
| `extractSocialHandles` | boolean | `true` | Handles from explicit social profile links only |
| `extractPgpContacts` | boolean | `true` | PGP fingerprint, key ID and contact email |

No cookies, API keys or proxy configuration are required or accepted.

#### Example: daily directory run (cheapest)

```json
{
    "mode": "directories",
    "source": "onionTree",
    "maxItems": 3,
    "verifyLiveness": false,
    "extractEmails": false,
    "extractPgpContacts": false
}
```

#### Example: merge all directories with liveness

```json
{
    "mode": "directories",
    "source": "all",
    "categories": ["news", "securedrop", "mail"],
    "maxItems": 50,
    "verifyLiveness": true,
    "livenessTimeoutSecs": 12,
    "includeUnreachable": true
}
```

#### Example: Ahmia index metadata

```json
{
    "mode": "search",
    "searchQueries": ["securedrop", "whistleblowing"],
    "maxResultsPerQuery": 10,
    "maxItems": 20
}
```

#### Example: read your own directory page

```json
{
    "mode": "byUrls",
    "startUrls": [{ "url": "https://example.org/onion-directory" }],
    "maxPages": 2,
    "maxItems": 25
}
```

### How liveness works

When `verifyLiveness` is true the actor starts the bundled Tor daemon and makes one GET request per unique onion domain, with a concurrency of four and a short jitter between requests. Example valid v3 entry: `http://bbcnewsd73hkzno2ini43t4gblxvycyac5aw4gnv7t2rccijh7745uqd.onion`. Results are classified as `live`, `challenge` (Cloudflare/DDoS-Guard or a 403/429), `timeout` or `unreachable`. After six consecutive transport failures the circuit breaker stops checking and marks the remaining entries `unknown`. A liveness failure never discards an already-built directory row.

### Limitations

- Legacy v2 onions are dead in modern Tor. They are validated out and counted as skipped; only 56-character v3 addresses are emitted.
- Directory data is stale by nature. Many listed services are offline; `status` is only meaningful when you enable `verifyLiveness`.
- dark.fail and tor.taxi do not expose category tags, so the category filter applies to OnionTree rows only. The marketplace denylist for untagged sources is heuristic and covers known market, drug-tracking, exploit-market and darknet-market-guide brands.
- Ahmia's search form uses a rotating hidden token and a session cookie. If the token cannot be read, search returns a clear status message instead of looping.
- OnionTree's own website can be unreachable; the actor reads its public GitHub tarball mirror instead.
- No actor can guarantee that a third-party directory entry is safe, lawful or current. Verify before use.

### FAQ

**Do I need to install Tor or configure a proxy?**
No. The container includes Tor. The actor starts it only when liveness is enabled or a `.onion` directory page is supplied.

**Does this actor crawl onion pages or collect page content?**
No. It reads directory pages and index-result metadata. A liveness check fetches only the root of an onion site to measure status and title; page bodies are not stored or emitted.

**Why are some directory entries missing?**
Marketplace and vendor-tagged OnionTree records are excluded by scope, payment tags are stripped from categories, legacy v2-only records are skipped, and duplicate domains are merged into one row. A source that is blocked or unreachable is skipped without failing the run.

**Why is a site marked `challenge` instead of `live`?**
The site answered with a Cloudflare/DDoS-Guard interstitial or a 403/429. It is reachable but not serving its normal page, so the actor reports the measured condition rather than assuming success.

**What does `sourceStatus` mean?**
It is dark.fail's own claim about a site. It is kept separate from the measured `status` because directory claims can disagree with a live Tor check.

**Why are PGP keys not included?**
Only the fingerprint, key ID and contact email are public contact metadata. The full public-key block is intentionally not stored.

**Can I search more than one keyword?**
Yes. `searchQueries` accepts up to 20 queries; each query is one Ahmia request, and `maxResultsPerQuery` plus `maxItems` bound the output.

**Why did an old task with `crawl` or `search` still run?**
Legacy input keys and mode names are mapped in code. `crawl` now means `byUrls`, `searchAndCrawl` now means `directories`, and `extractCryptoAddresses`/`extractApiKeys` are accepted but ignored with a status note because those data types are outside this actor's scope.

**Are empty fields returned as `null`?**
No. Fields with no value are omitted at every depth.

# Actor input Schema

## `mode` (type: `string`):

What to collect. `directories` aggregates public onion directory entries; `search` returns Ahmia index-result metadata; `byUrls` reads directory pages you supply. Legacy `crawl` and `searchAndCrawl` values are accepted and mapped in code.

## `source` (type: `string`):

Directory aggregation source (mode=directories). `all` merges OnionTree, dark.fail and tor.taxi.

## `categories` (type: `array`):

Keep OnionTree entries matching any selected category (mode=directories). Allowed values: archive, blockchain, blog, chat, dead, defcon, ebook, etherpad, forum, git, hacking, hosting, imageboard, jabber, journalism, keyserver, leaks, library, link-list, linux, mail, news, operating-system, pastebin, pgp, publishing, riseup, search, securedrop, security, social-media, tor, upload, video, vpn, vps, whistleblowing, wiki, xmpp. Empty means all neutral categories. This filter is ignored for dark.fail and tor.taxi rows because they carry no tags.

## `searchQueries` (type: `array`):

One or more keywords to query against the Ahmia index (mode=search). Each query is one request; result pages are never opened.

## `maxResultsPerQuery` (type: `integer`):

Cap on Ahmia rows per query (mode=search). The global maxItems cap still applies.

## `startUrls` (type: `array`):

Clearnet or v3 onion directory/index pages to read (mode=byUrls). Only entry links and adjacent structured contact links are parsed; linked onion pages are not opened.

## `maxItems` (type: `integer`):

Hard cap on records emitted to the dataset.

## `maxPages` (type: `integer`):

Maximum directory pages fetched in mode=byUrls.

## `verifyLiveness` (type: `boolean`):

Fetch each unique onion domain once through Tor and fill measured status, HTTP code, page title and check time. Applies to directories and byUrls; search mode never opens result pages.

## `livenessTimeoutSecs` (type: `integer`):

Per-domain timeout for the optional Tor liveness check.

## `includeUnreachable` (type: `boolean`):

When liveness is enabled, keep entries measured as timeout or unreachable. Applies only when verifyLiveness is true.

## `extractEmails` (type: `boolean`):

Include structured emails from OnionTree PGP user IDs and explicit mailto: links on directory pages. No free-text email mining.

## `extractPhones` (type: `boolean`):

Include phone numbers from explicit tel: links on directory pages only. No free-text phone mining.

## `extractSocialHandles` (type: `boolean`):

Include handles from explicit X/Twitter, Instagram and Telegram profile links on directory pages only.

## `extractPgpContacts` (type: `boolean`):

Include OnionTree PGP fingerprint, key ID and contact email. The public key block itself is never stored.

## Actor input object example

```json
{
  "mode": "directories",
  "source": "onionTree",
  "categories": [],
  "searchQueries": [],
  "maxResultsPerQuery": 3,
  "startUrls": [],
  "maxItems": 3,
  "maxPages": 1,
  "verifyLiveness": false,
  "livenessTimeoutSecs": 8,
  "includeUnreachable": true,
  "extractEmails": false,
  "extractPhones": false,
  "extractSocialHandles": false,
  "extractPgpContacts": false
}
```

# Actor output Schema

## `directoryEntries` (type: `string`):

Dataset containing directoryEntry rows (directories/byUrls) and searchResult rows (search).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "directories",
    "source": "onionTree",
    "categories": [],
    "searchQueries": [],
    "maxResultsPerQuery": 3,
    "startUrls": [],
    "maxItems": 3,
    "maxPages": 1,
    "verifyLiveness": false,
    "livenessTimeoutSecs": 8,
    "includeUnreachable": true,
    "extractEmails": false,
    "extractPhones": false,
    "extractSocialHandles": false,
    "extractPgpContacts": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("xyrelith-data/tor-hidden-service-crawler-and-contact-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "directories",
    "source": "onionTree",
    "categories": [],
    "searchQueries": [],
    "maxResultsPerQuery": 3,
    "startUrls": [],
    "maxItems": 3,
    "maxPages": 1,
    "verifyLiveness": False,
    "livenessTimeoutSecs": 8,
    "includeUnreachable": True,
    "extractEmails": False,
    "extractPhones": False,
    "extractSocialHandles": False,
    "extractPgpContacts": False,
}

# Run the Actor and wait for it to finish
run = client.actor("xyrelith-data/tor-hidden-service-crawler-and-contact-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "directories",
  "source": "onionTree",
  "categories": [],
  "searchQueries": [],
  "maxResultsPerQuery": 3,
  "startUrls": [],
  "maxItems": 3,
  "maxPages": 1,
  "verifyLiveness": false,
  "livenessTimeoutSecs": 8,
  "includeUnreachable": true,
  "extractEmails": false,
  "extractPhones": false,
  "extractSocialHandles": false,
  "extractPgpContacts": false
}' |
apify call xyrelith-data/tor-hidden-service-crawler-and-contact-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,xyrelith-data/tor-hidden-service-crawler-and-contact-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OL8Hv3EeydbTSbCFR/builds/UYaXoEvdhfwElKP1q/openapi.json