Dark Web Scraper - Tor .onion Page Extractor

What is Dark Web Scraper - Tor .onion Page Extractor?

Dark Web Scraper fetches the .onion, HTTP and HTTPS pages you supply through a bundled Tor client and returns one structured dataset row per page. Each row extracts the title, meta description, readable text preview, links with .onion flags, keyword hit counts, a SHA-256 content hash for change detection, response status and timing, and a redaction summary. Threat-intelligence analysts, brand-protection teams, OSINT researchers and AI agents use it to check known hidden-service pages, watch them for changes and pull their links, with sensitive-looking strings masked by default.

Why analysts choose this Tor page extractor

• Tor built in. routingMode: "tor" sends every request through the bundled Tor client; auto routes only .onion hosts through Tor; direct serves clearnet pages.
• Change detection. contentHash is computed on the readable text, so a scheduled check shows exactly when a page changes.
• Safety-first defaults. redactSensitive masks email-like, phone-like, crypto-wallet-like and secret-like strings, and the redaction object counts what was masked.
• Scoped link following. maxDepth and maxPages follow links inside the hosts you supply; crawlExternalHosts widens scope only when you choose.
• Keyword watch. keywordMatches returns case-insensitive counts for each phrase on each page.
• Diagnostics on hand. The key-value store holds SUMMARY, OUTPUT, RUN_CONFIG and TOR_DIAGNOSTICS, plus optional raw HTML_... snapshots.

Best fit for this Actor

• Best fit for lawful monitoring and research when you already know the pages you are authorized to inspect.
• Designed for brand and leak monitoring on known pages, uptime and change checks on hidden services, and link extraction inside a host.
• When you start from a keyword instead of a URL, start with Dark Web Search to find onion results, then pass the URLs you choose into startUrls here.
• For clearnet uptime and SSL checks on your own sites, use Website Uptime & SSL Checker instead.
• When a page names a username you want to trace across public platforms, continue with Sherlock Username OSINT.

Example workflow: from a watchlist to a change alert

A brand-protection analyst starts with five known .onion pages that have mentioned the company in the past. They add them to startUrls with routingMode set to tor, maxDepth at 0 and keywordMatches set to the brand name and product names. The dataset returns each page's contentHash, title, keywordMatches counts and statusCode. They schedule the same input daily, compare each contentHash with yesterday's value, and feed only changed pages with a non-zero keyword count into their case-management queue for human review.

Input

{
  "startUrls": [
    { "url": "https://check.torproject.org/" }
  ],
  "routingMode": "tor",
  "maxPages": 3,
  "maxDepth": 1,
  "crawlExternalHosts": false,
  "maxConcurrency": 2,
  "requestTimeoutSecs": 45,
  "keywordMatches": ["Tor Browser"],
  "responseFormat": "concise",
  "redactSensitive": true,
  "saveHtmlToKeyValueStore": false
}

Field	Type	What it controls
startUrls	array	Pages to fetch; http and https URLs, including .onion.
routingMode	string	tor (default), auto or direct.
maxPages	integer	Cap on in-scope pages. Default 3.
maxDepth	integer	Link-following depth. Default 1; 0 fetches only the supplied pages.
crawlExternalHosts	boolean	Follow links to other hosts. Default false.
maxConcurrency	integer	Parallel requests. Default 2, a Tor-friendly pace.
requestTimeoutSecs	integer	Per-page timeout. Default 45; raise it for slow onion services.
torBootstrapTimeoutSecs	integer	Wait time for the bundled Tor client. Default 75.
torProxyUrl	string	Optional external SOCKS endpoint such as socks5h://127.0.0.1:9050.
keywordMatches	array	Phrases to count on each page.
responseFormat	string	concise (default) or detailed, which adds text and markdown.
maxContentChars	integer	Text cap per page. Default 8000.
maxLinksPerPage	integer	Link objects stored per page. Default 75.
redactSensitive	boolean	Mask sensitive-looking strings. Default true.
saveHtmlToKeyValueStore	boolean	Save raw HTML under HTML_... keys. Default false.
userAgent	string	Optional custom user agent.

Output

Field	What it contains
url, rawUrl, finalUrl, route, depth, referrer	Requested URL, redirect target, Tor or direct route, and crawl position
ok, statusCode, statusMessage, contentType, responseTimeMs	Fetch status and timing
title, metaDescription, textPreview	Readable page signals, redacted by default
text, markdown	Longer content when responseFormat is detailed
contentChars, wordCount, truncated	Content size
contentHash	SHA-256 of the readable text, for change detection
links, linkCount, onionLinkCount, internalLinkCount, externalLinkCount	Extracted links with hostname, isOnion and crawlable flags
keywordMatches	Phrase counts for your keywords
redaction	Counts of masked email-like, phone-like, crypto-like and secret-like strings
htmlKey	Key of the saved raw HTML when that option is on
error	Diagnostic kind, message and suggestion for a page that did not load
fetchedAt	Collection time

{
  "url": "https://check.torproject.org/",
  "rawUrl": "https://check.torproject.org/",
  "finalUrl": "https://check.torproject.org/",
  "route": "tor",
  "depth": 0,
  "referrer": null,
  "ok": true,
  "statusCode": 200,
  "statusMessage": "OK",
  "contentType": "text/html; charset=utf-8",
  "responseTimeMs": 1386,
  "fetchedAt": "2026-09-10T12:55:13.326Z",
  "title": "Congratulations. This browser is configured to use Tor.",
  "metaDescription": null,
  "textPreview": "Congratulations. This browser is configured to use Tor.\nYour IP address appears to be: [REDACTED_PHONE]\nHowever, it does not appear to be Tor Browser.",
  "text": null,
  "markdown": null,
  "contentChars": 594,
  "wordCount": 103,
  "contentHash": "878cdbb635fab52b24ba0217501bbf1ec9f3520a77b56e677101552b54123e22",
  "truncated": false,
  "links": [
    {
      "url": "https://www.torproject.org/download/",
      "rawUrl": "https://www.torproject.org/download/",
      "text": "Click here to go to the download page",
      "hostname": "www.torproject.org",
      "isOnion": false,
      "crawlable": true
    }
  ],
  "linkCount": 9,
  "onionLinkCount": 0,
  "internalLinkCount": 0,
  "externalLinkCount": 9,
  "keywordMatches": [],
  "redaction": {
    "enabled": true,
    "emailLike": 0,
    "phoneLike": 1,
    "cryptoLike": 0,
    "secretLike": 0
  },
  "htmlKey": null,
  "error": null
}

Pricing

This Actor uses Pay per event with Apify platform usage included in the event price. The events are Actor Start (once per run) and Page scraped (one per page that is fetched, parsed and returned). Unreachable pages and diagnostic-only rows carry no page event. For example, a watchlist of 10 pages with maxDepth at 0 is one start event plus up to 10 page events. The live Pricing tab on this page shows the current price of each event; set a maximum charge in the run options for a firm spend cap.

API example

curl -X POST "https://api.apify.com/v2/acts/khadinakbar~dark-web-scraper/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": [{"url": "https://check.torproject.org/"}], "routingMode": "tor", "maxPages": 1, "maxDepth": 0}'

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("khadinakbar/dark-web-scraper").call(run_input={
    "startUrls": [{"url": "https://check.torproject.org/"}],
    "routingMode": "auto",
    "keywordMatches": ["Tor Browser"],
})
for page in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(page["url"], page["statusCode"], page["contentHash"], page["keywordMatches"])

Use with AI agents (MCP)

The Actor is available through Apify MCP for Claude, ChatGPT, Cursor and other MCP clients. Example prompt:

Fetch https://check.torproject.org/ through Tor with maxPages 1, then tell me the page title, the content hash, how many links it has, and whether anything was redacted.

The agent starts the Actor, reads the row with get-dataset-items, and reports title, contentHash, linkCount and the redaction counts. It can also read OUTPUT and TOR_DIAGNOSTICS from the key-value store for the outcome. Keep finalUrl and fetchedAt for provenance. For cost and scope, keep maxPages low and responseFormat on concise until the first rows look right.

Best results

Starting situation	Helpful action	Expected outcome
A list of known onion pages to watch	Set maxDepth to 0 and schedule daily	One hash per page to compare over time
Slow hidden services	Raise requestTimeoutSecs and keep maxConcurrency at 1 or 2	Steadier fetches through Tor
Mixed clearnet and onion URLs	Use routingMode: "auto"	Onion through Tor, clearnet direct
Text for an LLM summary	Set responseFormat to detailed	text and markdown on each row
Evidence for an investigation	Turn on saveHtmlToKeyValueStore	Raw HTML saved under htmlKey (unredacted, so store it carefully)

Hidden services come and go, so treat each row's ok and statusCode as the current reachability signal and re-check pages on a schedule.

Builder's note

I built this Actor for scoped monitoring rather than open-ended crawling: you supply the pages, and link following stays inside those hosts unless you widen it. I made redaction the default because onion pages often carry contact details and wallet strings that analysts rarely need verbatim, and I hash the readable text so a daily check turns into a simple did-it-change answer.

FAQ

Does it need Apify Proxy?

No. The Actor starts its own Tor client when Tor routing is on. You can also point it at your own SOCKS endpoint with torProxyUrl.

Can it fetch normal websites?

Yes. Use routingMode: "direct" for clearnet sites, or auto for a mixed list.

What if an onion page is slow to respond?

Raise requestTimeoutSecs, lower maxConcurrency, and check the row's diagnostic object and TOR_DIAGNOSTICS for the suggested next step.

Responsible use

Use this Actor only on pages you are legally allowed to access and analyze, and follow applicable laws and site terms. Some hidden-service content is harmful or sensitive, so keep scope tight, keep redaction on unless your investigation requires raw content, and store saved HTML securely. The Actor fetches public pages only and works without logins or access-control bypasses. Tor is a trademark of The Tor Project; this independent Actor is not affiliated with or endorsed by The Tor Project.