# Dark Web Scraper - Tor .onion Page Extractor (`khadinakbar/dark-web-scraper`) Actor Scrape supplied Tor .onion or HTTPS pages through bundled Tor. Extract readable text, links, keyword hits, content hashes, and diagnostics with redaction enabled by default. MCP-ready. - **URL**: https://apify.com/khadinakbar/dark-web-scraper.md - **Developed by:** [Khadin Akbar](https://apify.com/khadinakbar) (community) - **Categories:** Developer tools, Automation, MCP servers - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $6.00 / 1,000 page scrapeds This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Dark Web Scraper - Tor .onion Page Extractor **Dark Web Scraper** fetches supplied `.onion`, HTTP, or HTTPS pages through a bundled Tor client, extracts readable text, titles, metadata, links, keyword matches, content hashes, and diagnostics, and returns structured dataset rows that work well for Apify API and MCP clients. This actor is built for scoped, lawful OSINT and monitoring workflows. You provide the pages you are authorized to inspect. The actor does not search dark-web indexes, discover hidden services from keywords, log in to private systems, harvest secrets, or bypass access controls. ### What You Get | Output field | Meaning | | --- | --- | | `url`, `finalUrl`, `route` | Input URL, redirect target, and whether the request used Tor or direct routing | | `ok`, `statusCode`, `responseTimeMs` | Fetch status and timing | | `title`, `metaDescription`, `textPreview` | Readable page signals, redacted by default | | `text`, `markdown` | Longer extracted content when `responseFormat` is `detailed` | | `links` | Extracted links with hostnames, `.onion` flags, and crawlability flags | | `keywordMatches` | Exact keyword or phrase counts for each page | | `contentHash` | SHA-256 hash of readable text before redaction, useful for change detection | | `redaction` | Counts of email-like, phone-like, crypto-wallet-like, and secret-like strings masked | | `error` | Diagnostic kind, message, and suggestion for failures | The default key-value store also includes `SUMMARY`, `OUTPUT`, `RUN_CONFIG`, `TOR_DIAGNOSTICS`, and optional raw `HTML_...` snapshots. ### Common Use Cases - Brand, domain, or leak monitoring when you already have candidate pages. - Researching public `.onion` pages with reproducible fetch diagnostics. - Checking whether a known hidden-service page is online and has changed. - Extracting links from supplied pages while staying inside the original host scope. - Feeding an AI agent or MCP workflow with concise page records and safety-oriented defaults. ### What This Actor Does Not Do - It does not perform dark-web search by keyword. - It does not crawl arbitrary onion directories unless you provide those URLs and enable external-host crawling. - It does not log in, solve CAPTCHAs, or bypass access controls. - It does not provide a secret/API-key harvesting mode. - It does not guarantee that a Tor hidden service will be reachable; many are intermittent. ### Quick Start Run the actor with a supplied page: ```json { "startUrls": [ { "url": "https://check.torproject.org/" } ], "routingMode": "tor", "maxPages": 1, "maxDepth": 0, "responseFormat": "concise" } ```` Use direct routing for normal public websites: ```json { "startUrls": [ { "url": "https://example.com/" } ], "routingMode": "direct", "maxPages": 1, "maxDepth": 0, "keywordMatches": ["Example Domain"] } ``` Crawl same-host links from a supplied onion page: ```json { "startUrls": [ { "url": "http://exampleonionaddress.onion/" } ], "routingMode": "tor", "maxPages": 10, "maxDepth": 1, "crawlExternalHosts": false, "requestTimeoutSecs": 75 } ``` ### Inputs | Field | Default | Notes | | --- | --- | --- | | `startUrls` | Tor check page | Required URL list. Accepts `http` and `https` URLs, including `.onion`. | | `routingMode` | `tor` | `tor` sends all requests through Tor. `auto` sends only `.onion` through Tor. `direct` rejects `.onion`. | | `maxPages` | `3` | Hard cap on attempted in-scope pages, 1 to 100. | | `maxDepth` | `1` | Link-following depth, 0 to 3. | | `crawlExternalHosts` | `false` | Keeps crawling inside supplied hosts by default. | | `maxConcurrency` | `2` | Tor-friendly concurrency, 1 to 5. | | `requestTimeoutSecs` | `45` | Per-page timeout. Increase for slow onion services. | | `torBootstrapTimeoutSecs` | `75` | How long to wait for the bundled Tor client. | | `torProxyUrl` | empty | Optional external SOCKS endpoint, for example `socks5h://127.0.0.1:9050`. | | `keywordMatches` | `[]` | Exact phrase counts, case-insensitive. | | `responseFormat` | `concise` | `detailed` returns longer text and simple Markdown. | | `maxContentChars` | `8000` | Caps returned text per page. | | `maxLinksPerPage` | `75` | Caps stored link objects per page. | | `redactSensitive` | `true` | Masks sensitive-looking strings before content is stored. | | `saveHtmlToKeyValueStore` | `false` | Saves raw HTML under `HTML_...` keys. Raw HTML is not redacted. | ### Output Example ```json { "url": "https://check.torproject.org/", "finalUrl": "https://check.torproject.org/", "route": "tor", "depth": 0, "ok": true, "statusCode": 200, "responseTimeMs": 1842, "title": "Congratulations. This browser is configured to use Tor.", "textPreview": "Congratulations. This browser is configured to use Tor...", "contentChars": 743, "wordCount": 108, "contentHash": "b418...", "linkCount": 8, "onionLinkCount": 0, "keywordMatches": [ { "keyword": "Tor", "count": 4 } ], "redaction": { "enabled": true, "emailLike": 0, "phoneLike": 0, "cryptoLike": 0, "secretLike": 0 }, "error": null, "fetchedAt": "2026-07-08T12:00:00.000Z" } ``` ### Pricing This actor uses Pay Per Event pricing. | Event | Price | When charged | | --- | --- | --- | | `apify-actor-start` | `$0.00005` | Once when a run starts, scaled by memory | | `page-scraped` | `$0.006` | Each page that is successfully fetched, parsed, and returned | Failed pages, Tor bootstrap failures, invalid inputs, and diagnostic-only rows are not charged as `page-scraped`. Example costs: - 1 successful page: about `$0.00605` - 10 successful pages: about `$0.06005` - 100 successful pages: about `$0.60005` Use Apify's `maxTotalChargeUsd` run option if you want a hard spend cap. ### API Example ```bash curl -X POST "https://api.apify.com/v2/acts/khadinakbar~dark-web-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "startUrls": [{ "url": "https://check.torproject.org/" }], "routingMode": "tor", "maxPages": 1, "maxDepth": 0 }' ``` ### JavaScript Example ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_TOKEN' }); const run = await client.actor('khadinakbar/dark-web-scraper').call({ startUrls: [{ url: 'https://check.torproject.org/' }], routingMode: 'tor', maxPages: 1, maxDepth: 0, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items[0]); ``` ### MCP Usage When exposed through the Apify MCP server, the tool name is derived from the actor slug: ```text apify-actor-dark-web-scraper ``` For agent workflows, keep `maxPages` low, use `responseFormat: "concise"`, and inspect `OUTPUT` or the first dataset rows before requesting larger crawls. ### Legal And Safety Notes Only scrape pages you are legally allowed to access and analyze. Some dark-web content may be illegal, harmful, or sensitive. This actor is designed for bounded research and monitoring. It redacts sensitive-looking content by default, keeps crawling inside supplied hosts unless you opt out, and reports diagnostics instead of guessing. ### Related Actors - [Website Uptime Monitor](https://apify.com/khadinakbar/website-uptime-monitor) - [Web Unlocker](https://apify.com/khadinakbar/web-unlocker) - [Username OSINT Scraper](https://apify.com/khadinakbar/username-osint-scraper) - [Google SERP All-in-One Scraper](https://apify.com/khadinakbar/google-serp-all-in-one-scraper) - [Broken Link Checker](https://apify.com/khadinakbar/broken-link-checker) - [SEO Domain Keyword Scraper](https://apify.com/khadinakbar/seo-domain-keyword-scraper) - [Bulk Website Contact Extractor](https://apify.com/khadinakbar/bulk-website-contact-extractor) - [Contact Details Scraper](https://apify.com/khadinakbar/contact-details-scraper) - [Reddit Posts Search Scraper](https://apify.com/khadinakbar/reddit-posts-search-scraper) - [AI Brand Monitor MCP](https://apify.com/khadinakbar/ai-brand-monitor-mcp) ### FAQ #### Can I use this as a dark-web search engine? No. This actor fetches supplied URLs and optional in-scope links. It does not search hidden-service indexes by keyword. #### Does it need Apify Proxy? No. The actor starts a Tor client inside the container when Tor routing is needed. You can also provide your own `torProxyUrl`. #### Can it scrape normal websites? Yes. Use `routingMode: "direct"` for clearnet sites, or `auto` if your URL list contains a mix of clearnet and `.onion` URLs. #### Why did my onion URL fail? Hidden services can be offline, slow, blocked, or intermittently unreachable. Check `error.kind`, `error.suggestion`, and `TOR_DIAGNOSTICS`, then retry with a higher timeout and lower concurrency. #### Can it save raw HTML? Yes. Set `saveHtmlToKeyValueStore` to `true`. Raw HTML is saved under `HTML_...` keys and is not redacted, so use it only for authorized investigations. #### Can I integrate it with another app? Yes. Use the Apify API, Apify client libraries, webhooks, datasets, or MCP. The default output is structured JSON. #### How much can I scrape? The input schema caps `maxPages` at 100 per run. This keeps Tor runs predictable and prevents accidental broad crawls. Start small and increase gradually. #### How can I share feedback? Open an issue from the Apify actor page or contact the actor owner with a sample run ID and the URL pattern that failed. # Actor input Schema ## `startUrls` (type: `array`): Use this when you already know the pages to inspect. Accepts http or https URLs such as http://exampleonionaddress.onion/ or https://check.torproject.org/. Defaults to a Tor check page. This is not a keyword search or hidden-service discovery input. ## `routingMode` (type: `string`): Use this to choose how requests leave the actor. auto sends .onion URLs through Tor and clearnet URLs directly; tor sends all URLs through Tor; direct sends only clearnet URLs directly. Defaults to tor for a privacy-preserving health check. This is not a proxy bypass tool for login-only sites. ## `maxPages` (type: `integer`): Use this to cap the total number of pages fetched across all seeds. Accepts 1 to 100; default is 3 and prefill is 3. The actor stops before this cap and will not charge beyond successful pages. This is not a search-results limit. ## `maxDepth` (type: `integer`): Use this to control link following from each seed page. Accepts 0 to 3; depth 0 fetches only the supplied URLs. Defaults to 1. This does not discover links from external search engines or onion directories. ## `crawlExternalHosts` (type: `boolean`): Use this only when you want links to other hosts added to the crawl queue. Defaults to false so the actor stays inside the supplied host scope. The output still reports external links. This is not a broad dark-web discovery mode. ## `maxConcurrency` (type: `integer`): Use this to tune how many pages are fetched at once. Accepts 1 to 5; default is 2 because Tor hidden services are often slow. Higher values may time out more often. This is not a retry count. ## `requestTimeoutSecs` (type: `integer`): Use this to cap each page fetch. Accepts 10 to 120 seconds; default is 45. Onion services can be slow, so short timeouts may produce diagnostic rows. This is not the total actor timeout. ## `torBootstrapTimeoutSecs` (type: `integer`): Use this to cap how long the bundled Tor daemon may take to become ready. Accepts 20 to 180 seconds; default is 75. Raise it if Tor circuits are slow in the cloud. This is ignored when routingMode is direct. ## `torProxyUrl` (type: `string`): Use this when you want to supply an existing Tor SOCKS endpoint instead of starting the bundled Tor daemon. Accepts a URL like socks5h://127.0.0.1:9050. Defaults to empty, which starts bundled Tor when needed. This is not an HTTP proxy URL. ## `keywordMatches` (type: `array`): Use this to count exact keyword or phrase matches in each page. Accepts one keyword per line, such as your brand, domain, or product name. Defaults to an empty list. This does not search the dark web for new pages. ## `responseFormat` (type: `string`): Use this to control how much page content appears in each dataset item. concise returns a preview only; detailed includes longer text and simple Markdown. Defaults to concise for MCP clients. This does not affect link or diagnostic fields. ## `maxContentChars` (type: `integer`): Use this to limit returned text size per page. Accepts 500 to 50000 characters; default is 8000. Longer content is truncated with a note. This does not increase the page fetch size cap. ## `maxLinksPerPage` (type: `integer`): Use this to limit how many extracted links are stored per page. Accepts 0 to 250; default is 75. Link counts still reflect all discovered links. This is not the number of links crawled. ## `redactSensitive` (type: `boolean`): Use this to mask email-like, phone-like, crypto-wallet-like, and API-key-like strings before content is saved. Defaults to true for safer research workflows. Turn off only for lawful authorized investigations. This does not delete the source page. ## `saveHtmlToKeyValueStore` (type: `boolean`): Use this when you need page snapshots for audit or reproducibility. Defaults to false because raw HTML can contain sensitive material. When true, HTML is saved under HTML\_ keys and referenced from dataset rows. This does not redact the raw HTML. ## `userAgent` (type: `string`): Use this to override the HTTP User-Agent header. Accepts a normal browser user agent string. Defaults to a current desktop browser-like value. This is not a browser fingerprint or JavaScript execution setting. ## Actor input object example ```json { "startUrls": [ { "url": "https://check.torproject.org/" } ], "routingMode": "tor", "maxPages": 3, "maxDepth": 1, "crawlExternalHosts": false, "maxConcurrency": 2, "requestTimeoutSecs": 45, "torBootstrapTimeoutSecs": 75, "keywordMatches": [], "responseFormat": "concise", "maxContentChars": 8000, "maxLinksPerPage": 75, "redactSensitive": true, "saveHtmlToKeyValueStore": false } ``` # Actor output Schema ## `results` (type: `string`): One structured record per fetched page or diagnostic fetch attempt. ## `summary` (type: `string`): Summary with counts, route mode, Tor readiness, charged pages, and warnings. ## `output` (type: `string`): Compact JSON summary intended for API and MCP clients. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://check.torproject.org/" } ], "routingMode": "tor", "maxPages": 3, "maxDepth": 1 }; // Run the Actor and wait for it to finish const run = await client.actor("khadinakbar/dark-web-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://check.torproject.org/" }], "routingMode": "tor", "maxPages": 3, "maxDepth": 1, } # Run the Actor and wait for it to finish run = client.actor("khadinakbar/dark-web-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://check.torproject.org/" } ], "routingMode": "tor", "maxPages": 3, "maxDepth": 1 }' | apify call khadinakbar/dark-web-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=khadinakbar/dark-web-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Dark Web Scraper - Tor .onion Page Extractor", "description": "Scrape supplied Tor .onion or HTTPS pages through bundled Tor. Extract readable text, links, keyword hits, content hashes, and diagnostics with redaction enabled by default. MCP-ready.", "version": "0.1", "x-build-id": "fSRzgKsGXj37NkZ4H" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/khadinakbar~dark-web-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-khadinakbar-dark-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/khadinakbar~dark-web-scraper/runs": { "post": { "operationId": "runs-sync-khadinakbar-dark-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/khadinakbar~dark-web-scraper/run-sync": { "post": { "operationId": "run-sync-khadinakbar-dark-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Start URLs", "type": "array", "description": "Use this when you already know the pages to inspect. Accepts http or https URLs such as http://exampleonionaddress.onion/ or https://check.torproject.org/. Defaults to a Tor check page. This is not a keyword search or hidden-service discovery input.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "routingMode": { "title": "Routing mode", "enum": [ "tor", "auto", "direct" ], "type": "string", "description": "Use this to choose how requests leave the actor. auto sends .onion URLs through Tor and clearnet URLs directly; tor sends all URLs through Tor; direct sends only clearnet URLs directly. Defaults to tor for a privacy-preserving health check. This is not a proxy bypass tool for login-only sites.", "default": "tor" }, "maxPages": { "title": "Max pages", "minimum": 1, "maximum": 100, "type": "integer", "description": "Use this to cap the total number of pages fetched across all seeds. Accepts 1 to 100; default is 3 and prefill is 3. The actor stops before this cap and will not charge beyond successful pages. This is not a search-results limit.", "default": 3 }, "maxDepth": { "title": "Max crawl depth", "minimum": 0, "maximum": 3, "type": "integer", "description": "Use this to control link following from each seed page. Accepts 0 to 3; depth 0 fetches only the supplied URLs. Defaults to 1. This does not discover links from external search engines or onion directories.", "default": 1 }, "crawlExternalHosts": { "title": "Crawl external hosts", "type": "boolean", "description": "Use this only when you want links to other hosts added to the crawl queue. Defaults to false so the actor stays inside the supplied host scope. The output still reports external links. This is not a broad dark-web discovery mode.", "default": false }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "maximum": 5, "type": "integer", "description": "Use this to tune how many pages are fetched at once. Accepts 1 to 5; default is 2 because Tor hidden services are often slow. Higher values may time out more often. This is not a retry count.", "default": 2 }, "requestTimeoutSecs": { "title": "Request timeout", "minimum": 10, "maximum": 120, "type": "integer", "description": "Use this to cap each page fetch. Accepts 10 to 120 seconds; default is 45. Onion services can be slow, so short timeouts may produce diagnostic rows. This is not the total actor timeout.", "default": 45 }, "torBootstrapTimeoutSecs": { "title": "Tor bootstrap timeout", "minimum": 20, "maximum": 180, "type": "integer", "description": "Use this to cap how long the bundled Tor daemon may take to become ready. Accepts 20 to 180 seconds; default is 75. Raise it if Tor circuits are slow in the cloud. This is ignored when routingMode is direct.", "default": 75 }, "torProxyUrl": { "title": "External Tor SOCKS proxy", "type": "string", "description": "Use this when you want to supply an existing Tor SOCKS endpoint instead of starting the bundled Tor daemon. Accepts a URL like socks5h://127.0.0.1:9050. Defaults to empty, which starts bundled Tor when needed. This is not an HTTP proxy URL." }, "keywordMatches": { "title": "Keyword matches", "type": "array", "description": "Use this to count exact keyword or phrase matches in each page. Accepts one keyword per line, such as your brand, domain, or product name. Defaults to an empty list. This does not search the dark web for new pages.", "default": [], "items": { "type": "string" } }, "responseFormat": { "title": "Response format", "enum": [ "concise", "detailed" ], "type": "string", "description": "Use this to control how much page content appears in each dataset item. concise returns a preview only; detailed includes longer text and simple Markdown. Defaults to concise for MCP clients. This does not affect link or diagnostic fields.", "default": "concise" }, "maxContentChars": { "title": "Max content characters", "minimum": 500, "maximum": 50000, "type": "integer", "description": "Use this to limit returned text size per page. Accepts 500 to 50000 characters; default is 8000. Longer content is truncated with a note. This does not increase the page fetch size cap.", "default": 8000 }, "maxLinksPerPage": { "title": "Max links per page", "minimum": 0, "maximum": 250, "type": "integer", "description": "Use this to limit how many extracted links are stored per page. Accepts 0 to 250; default is 75. Link counts still reflect all discovered links. This is not the number of links crawled.", "default": 75 }, "redactSensitive": { "title": "Redact sensitive patterns", "type": "boolean", "description": "Use this to mask email-like, phone-like, crypto-wallet-like, and API-key-like strings before content is saved. Defaults to true for safer research workflows. Turn off only for lawful authorized investigations. This does not delete the source page.", "default": true }, "saveHtmlToKeyValueStore": { "title": "Save raw HTML to KV store", "type": "boolean", "description": "Use this when you need page snapshots for audit or reproducibility. Defaults to false because raw HTML can contain sensitive material. When true, HTML is saved under HTML_ keys and referenced from dataset rows. This does not redact the raw HTML.", "default": false }, "userAgent": { "title": "User agent", "type": "string", "description": "Use this to override the HTTP User-Agent header. Accepts a normal browser user agent string. Defaults to a current desktop browser-like value. This is not a browser fingerprint or JavaScript execution setting." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```