# Residential Web Scraper (US Verizon IPs) (`exotic_integrity/residential-web-scraper`) Actor Scrape any URL through 23 real US residential IPs (Verizon AS701). Bring your own MoneyMaker API key. - **URL**: https://apify.com/exotic\_integrity/residential-web-scraper.md - **Developed by:** [Kevin Yen](https://apify.com/exotic_integrity) (community) - **Categories:** Developer tools - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Residential Web Scraper — 23 Verizon US IPs Fetch any URL through a real US residential IP (Verizon AS701, no datacenter fingerprint). Each call rotates LRU through a 23-IP pool. Every dataset item includes the literal egress IP so you can verify it yourself with `ipapi.com` or WHOIS. This Actor is a thin wrapper around the **MoneyMaker Residential Scraping API**. Apify hosts and runs the Actor; the residential proxy fleet runs on Hetzner with Verizon AS701 ISP proxies. Apify does **not** charge for this Actor — you bring your own API key from the MoneyMaker listing on RapidAPI (or directly from the publisher). --- ### Why residential matters Most modern target sites (Amazon, Zillow, LinkedIn, pricing pages, news index pages) apply ASN-level filtering at the TCP layer before the page renders. Datacenter ranges (AWS, GCP, DO, Azure) get a stripped HTML response or a challenge. A Verizon AS701 IP is in the same ASN class as 30M+ US home-broadband subscribers, so the request looks like a shopper on home Wi-Fi. Concrete differences against datacenter origins: - **Pricing pages**: Notion, Vercel, Linear serve different HTML to AWS visitors than to home broadband. ASN gating happens before React hydrates. - **E-commerce**: Amazon's "Robot Check" interstitial fires on roughly 100% of bare AWS/GCP requests. Residential gets the live Buy Box price. - **Job boards**: LinkedIn returns a stripped guest skeleton or a 999 to datacenter ranges; residential gets the full Apollo state JSON. - **Real estate**: Zillow / Redfin use Akamai Bot Manager — datacenter ASNs hard-fail at the TCP layer. Residential gets the full `__NEXT_DATA__`. --- ### Inputs | Field | Type | Required | Description | | ----------------- | ------------ | -------- | ------------------------------------------------------------ | | `urls` | array | yes | One or more http(s) URLs to fetch | | `apiKey` | string (secret) | yes | Your MoneyMaker API key (`mm_sk_...`) | | `apiBaseUrl` | string | no | Default: the public MM origin | | `method` | string enum | no | GET / POST / HEAD (default GET) | | `timeoutS` | integer | no | 3-60s (default 20) | | `followRedirects` | boolean | no | Default true | | `extraHeaders` | object | no | Forwarded on every request (e.g. User-Agent) | | `concurrency` | integer | no | 1-20 parallel fetches (default 5) | | `verifyTls` | boolean | no | Default true | --- ### Output One dataset item per input URL. Successful items: ```json { "url": "https://example.com/", "ok": true, "request_id": "req_w-v8wa1qZ79YcihQ", "status_code": 200, "ip_used": "65.195.39.208", "proxy_node": "Hetzner-12", "headers": { "content-type": "text/html; charset=UTF-8" }, "body": "...", "body_truncated": false, "upstream_latency_ms": 1313, "actor_latency_ms": 1380 } ```` Failed items keep the URL and an `error` string for retry triage: ```json { "url": "https://broken.example/", "ok": false, "upstream_status": 502, "error": "upstream_http_502", "upstream_body_excerpt": "{\"error\":\"upstream_proxy_error: ...\"}", "actor_latency_ms": 1820 } ``` The Actor's key-value store also gets a `SUMMARY` record with totals and the distinct residential IPs seen during the run. *** ### How it works ``` Apify run │ ▼ Actor (this code) │ POST /v1/scrape, Bearer ▼ MoneyMaker Scraping API (37.27.176.80:8443) │ Picks an LRU residential proxy from the 23-IP pool ▼ Verizon AS701 home-broadband IP │ GET / POST your target URL ▼ Target site responds → JSON body returned to Actor → push_data ``` You only pay MoneyMaker for the underlying calls — Apify is free. Costs follow the MoneyMaker pricing tiers on RapidAPI: 1,000 free / month, then $19 / 50k, $49 / 250k, $149 / 1M. *** ### Pacing guidance - Keep < 1 req/sec per target site averaged over 60s. The pool absorbs bursts but individual sites profile cadence over 5-60 minute windows. - 23 unique IPs in the rotation handle moderate concurrency cleanly. Going above `concurrency: 10` mostly trades parallelism for per-IP request density at one site, which is what you don't want. - Cache aggressively. A pricing page changes once a day at most. - Respect robots.txt and ToS. None of the Actor's behavior is designed to scrape paid content or bypass authentication. *** ### Limitations - **No JS rendering.** Static HTML only. For sites that need a headless browser, use Apify's Web Scraper Actor or ScrapingBee. - **US only.** All egress is Verizon AS701 / United States. No EU / APAC residential pool. - **No CAPTCHA solving.** If the target shows a CAPTCHA, you'll get the CAPTCHA HTML in the body — it's not auto-solved. - **Body cap of 2 MB** per response (truncated and flagged via `body_truncated: true`). *** ### Source & support The Actor source lives at [`sksyen/moneymaker-dashboard/scripts/apify_actor`](https://github.com/sksyen/moneymaker-dashboard). Issues and feature requests go on the GitHub repo. Underlying API contract is published as OpenAPI 3.0 at the `/openapi.yaml` path on the origin. # Actor input Schema ## `urls` (type: `array`): One or more http(s) URLs. Each URL is fetched through a randomly-rotated US residential IP from our 23-IP Verizon AS701 pool. Each fetched page becomes one item in the dataset. ## `apiKey` (type: `string`): Your API key from the MoneyMaker Residential Scraping API (sign up at the listing on RapidAPI or contact for direct key). Format: mm\_sk\_... — paste the bare token, not the Bearer header. ## `apiBaseUrl` (type: `string`): Public endpoint of the Residential Scraping API. Provided by your API key issuer. ## `method` (type: `string`): Only GET, POST, and HEAD are supported by the upstream API. ## `timeoutS` (type: `integer`): Upstream allows 3-60s. Default 20s is fine for most static pages. ## `followRedirects` (type: `boolean`): Whether the upstream API should follow HTTP 301/302 redirects automatically. Default true; disable to capture redirect responses verbatim. ## `extraHeaders` (type: `object`): Optional object of header-name to header-value pairs forwarded on every request (e.g. User-Agent, Accept-Language). ## `concurrency` (type: `integer`): How many URLs to fetch in parallel. Stay <= 10 to be polite — the upstream pool has 23 IPs and steady-state throughput is ~8-11 req/s. ## `verifyTls` (type: `boolean`): Disable only if pointing at the bare-IP HTTP origin without a valid cert. Default true. ## Actor input object example ```json { "urls": [ { "url": "https://api.ipify.org?format=json" } ], "apiBaseUrl": "https://api.example.com:8443", "method": "GET", "timeoutS": 20, "followRedirects": true, "concurrency": 5, "verifyTls": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ { "url": "https://api.ipify.org?format=json" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("exotic_integrity/residential-web-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [{ "url": "https://api.ipify.org?format=json" }] } # Run the Actor and wait for it to finish run = client.actor("exotic_integrity/residential-web-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ { "url": "https://api.ipify.org?format=json" } ] }' | apify call exotic_integrity/residential-web-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=exotic_integrity/residential-web-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Residential Web Scraper (US Verizon IPs)", "description": "Scrape any URL through 23 real US residential IPs (Verizon AS701). Bring your own MoneyMaker API key.", "version": "0.1", "x-build-id": "bnLXjmDQLwmXWOR8k" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/exotic_integrity~residential-web-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-exotic_integrity-residential-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/exotic_integrity~residential-web-scraper/runs": { "post": { "operationId": "runs-sync-exotic_integrity-residential-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/exotic_integrity~residential-web-scraper/run-sync": { "post": { "operationId": "run-sync-exotic_integrity-residential-web-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "urls", "apiKey" ], "properties": { "urls": { "title": "URLs to scrape", "type": "array", "description": "One or more http(s) URLs. Each URL is fetched through a randomly-rotated US residential IP from our 23-IP Verizon AS701 pool. Each fetched page becomes one item in the dataset.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "apiKey": { "title": "MoneyMaker API key", "type": "string", "description": "Your API key from the MoneyMaker Residential Scraping API (sign up at the listing on RapidAPI or contact for direct key). Format: mm_sk_... — paste the bare token, not the Bearer header." }, "apiBaseUrl": { "title": "API base URL", "type": "string", "description": "Public endpoint of the Residential Scraping API. Provided by your API key issuer.", "default": "https://api.example.com:8443" }, "method": { "title": "HTTP method", "enum": [ "GET", "POST", "HEAD" ], "type": "string", "description": "Only GET, POST, and HEAD are supported by the upstream API.", "default": "GET" }, "timeoutS": { "title": "Per-request timeout (seconds)", "minimum": 3, "maximum": 60, "type": "integer", "description": "Upstream allows 3-60s. Default 20s is fine for most static pages.", "default": 20 }, "followRedirects": { "title": "Follow redirects", "type": "boolean", "description": "Whether the upstream API should follow HTTP 301/302 redirects automatically. Default true; disable to capture redirect responses verbatim.", "default": true }, "extraHeaders": { "title": "Extra request headers", "type": "object", "description": "Optional object of header-name to header-value pairs forwarded on every request (e.g. User-Agent, Accept-Language)." }, "concurrency": { "title": "Parallel requests", "minimum": 1, "maximum": 20, "type": "integer", "description": "How many URLs to fetch in parallel. Stay <= 10 to be polite — the upstream pool has 23 IPs and steady-state throughput is ~8-11 req/s.", "default": 5 }, "verifyTls": { "title": "Verify TLS to upstream", "type": "boolean", "description": "Disable only if pointing at the bare-IP HTTP origin without a valid cert. Default true.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```