# πŸš€ Web Traffic Generator (`scrapier/web-traffic-generator`) Actor - **URL**: https://apify.com/scrapier/web-traffic-generator.md - **Developed by:** [Scrapier](https://apify.com/scrapier) (community) - **Categories:** Automation, Developer tools, SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $4.99 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## πŸš€ Web Traffic Generator β€” Real Browser Visits **Web Traffic Generator** sends real, human-like browser traffic to any website you choose. Every visit launches a genuine Chromium browser, emulates a random desktop / mobile / tablet device, scrolls the page naturally, and (optionally) crawls internal links β€” so the traffic looks exactly like a real person browsing your site. A built-in **auto-escalating proxy ladder** keeps visits flowing even when a site rate-limits or blocks the request. Perfect for warming up analytics dashboards, load-checking pages, testing A/B variants, demoing funnels, and stress-testing how a site behaves under realistic, multi-device traffic. --- ### ✨ Why Choose This Actor? | | This Actor | Typical alternatives | |---|---|---| | 🌐 Proxy strategy | **Auto-escalating ladder** β€” direct β†’ datacenter β†’ residential | Single fixed proxy | | πŸ“± Devices | 8 real device profiles (desktop / mobile / tablet) | Desktop-only or none | | πŸ•ΈοΈ Crawling | Follows internal links, multi-page visits | Single-page only | | πŸ“Š Output | 4 ready-made dataset views β€” overview, devices, crawl, diagnostics | Flat dump | | πŸ’Ύ Live results | Every visit saved the moment it finishes | Saved only at the end | | πŸ›‘οΈ Reliability | Retries, backoff, block detection, graceful partial output | Fails on first block | --- ### πŸ”‘ Key Features - **πŸ”— Bulk URLs** β€” paste one or hundreds of websites; visits are spread across them. - **πŸ§‘β€πŸ€β€πŸ§‘ Concurrency control** β€” run many visits in parallel for fast, high-volume traffic. - **πŸ“± Real device emulation** β€” Windows, macOS, Linux, Pixel, iPhone, Galaxy, iPad, Galaxy Tab. - **βŒ› Natural behavior** β€” smooth, randomized scrolling and dwell time on every page. - **πŸ•ΈοΈ Deep crawling** β€” each visit can browse internal pages, same-domain or site-wide. - **🌐 Smart proxy ladder** β€” starts direct, escalates to datacenter then residential **only when blocked**, and ratchets to the higher tier for the rest of the run. - **πŸ” Resilient** β€” block detection, retries with exponential backoff, and per-visit error capture. - **πŸ“Š Live, sectioned output** β€” results stream to the dataset in real time, split into four views. --- ### πŸ“₯ Input | Field | Type | Description | Default | |-------|------|-------------|---------| | `startUrls` | array | Website URLs to visit (bulk supported). | β€” (required) | | `numberOfVisitors` | integer | Total visits to generate across all URLs. | `10` | | `simultaneousVisitors` | integer | Visits running in parallel. | `5` | | `stayDurationSeconds` | integer | Seconds each visit scrolls a page. | `10` | | `useDesktop` | boolean | Emulate desktop browsers. | `true` | | `useMobile` | boolean | Emulate mobile phones. | `true` | | `useTablet` | boolean | Emulate tablets. | `true` | | `crawlWebsite` | boolean | Follow internal links and visit more pages. | `true` | | `crawlMaxPages` | integer | Max extra pages per crawling visit. | `20` | | `crawlSameDomainOnly` | boolean | Keep crawling on the start URL's domain. | `true` | | `proxyConfiguration` | object | Proxy used for the fallback tiers. | `{ "useApifyProxy": true }` | | `maxRetries` | integer | Residential retry attempts before giving up. | `3` | | `retryBackoffSeconds` | integer | Base wait between residential retries. | `2` | | `navigationTimeoutMs` | integer | Page-load timeout in milliseconds. | `90000` | #### Example input ```json { "startUrls": [{ "url": "https://bebity.io" }], "numberOfVisitors": 25, "simultaneousVisitors": 5, "stayDurationSeconds": 12, "useDesktop": true, "useMobile": true, "useTablet": true, "crawlWebsite": true, "crawlMaxPages": 10, "crawlSameDomainOnly": true, "proxyConfiguration": { "useApifyProxy": true } } ```` *** ### πŸ“€ Output Each visit is pushed to the dataset the instant it finishes. One record per visit: ```json { "url": "https://bebity.io", "success": true, "duration": 18664, "durationSeconds": 18.66, "startTime": 1779343743840, "endTime": 1779343762504, "visitedAt": "2026-05-21T08:09:03.840000+00:00", "deviceType": "mobile", "deviceLabel": "iPhone (iOS)", "deviceUserAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 16_6 ...) Safari/604.1", "deviceViewport": "390x664", "device": { "userAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 16_6 ...) Safari/604.1", "viewport": { "width": 390, "height": 664 }, "screen": { "width": 390, "height": 844 }, "deviceScaleFactor": 3, "isMobile": true, "hasTouch": true, "defaultBrowserType": "chromium" }, "proxyTier": "none", "proxyUsed": false, "proxyUrl": null, "crawlEnabled": true, "crawledPagesCount": 0, "crawledPages": [], "pagesVisited": 1, "attempts": 1, "blockEvents": 0, "error": null } ``` | Field | Meaning | |-------|---------| | `success` | Whether the visit completed without being blocked. | | `duration` / `durationSeconds` | Total visit time (ms / seconds). | | `device` | Full emulated device profile. | | `proxyTier` | Tier the visit ended on β€” `none`, `datacenter` or `residential`. | | `proxyUrl` | Proxy endpoint used (password redacted). | | `crawledPages` | Internal pages browsed during the visit. | | `attempts` / `blockEvents` | Retry count and number of blocks hit. | | `error` | Failure reason, if the visit did not succeed. | The dataset ships with **four output views** so results are easy to read section by section: **🌐 Visit Overview**, **πŸ“± Device Breakdown**, **πŸ•ΈοΈ Crawled Pages**, and **πŸ›‘οΈ Proxy & Diagnostics**. A `RUN_SUMMARY` is also saved to the key-value store. *** ### πŸš€ How to Use (Apify Console) 1. Log in at [console.apify.com](https://console.apify.com) β†’ **Actors**. 2. Open **Web Traffic Generator**. 3. Add your website URLs and tune the visit count, devices, and crawl options. 4. Click **Start**. 5. Watch the live log β€” every visit reports its result in real time. 6. Open the **Output** tab and switch between the four views. 7. Export to JSON / CSV / XLSX. ### πŸ€– Use via API ```bash curl -X POST "https://api.apify.com/v2/acts//runs?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "startUrls": [{ "url": "https://bebity.io" }], "numberOfVisitors": 25, "simultaneousVisitors": 5, "proxyConfiguration": { "useApifyProxy": true } }' ``` *** ### 🎯 Best Use Cases - πŸ“ˆ Warming up website analytics and dashboards with realistic multi-device traffic. - πŸ§ͺ A/B test and funnel demos that need genuine browser sessions. - βš™οΈ Smoke / load checks on landing pages and marketing sites. - 🌍 Generating geo-distributed traffic via the proxy ladder. ### πŸ’° Pricing This actor runs on the **pay-per-event** model. You are billed for actor startup and for each **successful visit** (`page-visit` event). Failed or blocked visits are not charged. Underlying platform usage (compute, proxies) follows your Apify plan. Exact event prices are shown on the actor's Store page before you run it. ### ❓ FAQ **Does it really load pages like a person?** Yes β€” every visit is a real Chromium browser with an emulated device, randomized scrolling, and dwell time. **What happens if a site blocks me?** The actor detects the block and escalates: direct β†’ datacenter proxy β†’ residential proxy. Once it escalates, the higher tier is reused for all remaining visits. **Why did a visit fail?** Check the `error` field and the **Proxy & Diagnostics** view. Common causes are unreachable sites or aggressive anti-bot protection. **Can I raise concurrency?** Yes β€” but each parallel visit is a full browser. Increase the run's memory when setting `simultaneousVisitors` above 5. ### πŸ›Ÿ Support & Feedback Found a bug or want a feature? Open an issue on the actor's **Issues** tab in the Apify Console. ### βš–οΈ Legal Use this actor only on websites you own or are authorized to test. You are responsible for complying with the target site's Terms of Service and all applicable laws. # Actor input Schema ## `startUrls` (type: `array`): One or more website URLs to send visits to. Paste them in bulk β€” every visit picks one of these URLs in round-robin order. A URL without http(s):// is treated as https://. ## `numberOfVisitors` (type: `integer`): Total number of browser visits to generate across all URLs. ## `simultaneousVisitors` (type: `integer`): How many visits run in parallel. Higher values finish faster but need more memory β€” increase the run's RAM if you set this above 5. ## `stayDurationSeconds` (type: `integer`): How long each visit stays on a page, scrolling naturally, before moving on. ## `useDesktop` (type: `boolean`): Emulate desktop browsers (Windows, macOS, Linux). ## `useMobile` (type: `boolean`): Emulate mobile phones (Pixel, iPhone, Galaxy). ## `useTablet` (type: `boolean`): Emulate tablets (iPad, Galaxy Tab). ## `crawlWebsite` (type: `boolean`): When enabled, every visit also follows internal links and browses additional pages of the site. ## `crawlMaxPages` (type: `integer`): Maximum number of extra pages a single crawling visit will browse. ## `crawlSameDomainOnly` (type: `boolean`): Restrict crawling to pages on the start URL's domain. Disable to also follow external links. ## `proxyConfiguration` (type: `object`): Proxy used for the fallback tiers. Every visit begins with a direct connection; if a site blocks it, the actor escalates automatically β€” first to a datacenter proxy, then to a residential proxy β€” and sticks with the higher tier for the rest of the run. ## `maxRetries` (type: `integer`): How many times to retry a blocked visit on the residential proxy before giving up on it. ## `retryBackoffSeconds` (type: `integer`): Base wait time between residential retry attempts. The wait grows with each attempt. ## `navigationTimeoutMs` (type: `integer`): How long to wait for a page to load before treating the visit as failed. Residential proxies often need 30–60 seconds. ## `headless` (type: `boolean`): Run browsers in headless mode. Always keep enabled on the Apify platform. ## Actor input object example ```json { "startUrls": [ "https://bebity.io" ], "numberOfVisitors": 10, "simultaneousVisitors": 5, "stayDurationSeconds": 10, "useDesktop": true, "useMobile": true, "useTablet": true, "crawlWebsite": true, "crawlMaxPages": 20, "crawlSameDomainOnly": true, "proxyConfiguration": { "useApifyProxy": true }, "maxRetries": 3, "retryBackoffSeconds": 2, "navigationTimeoutMs": 90000, "headless": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ "https://bebity.io" ], "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("scrapier/web-traffic-generator").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": ["https://bebity.io"], "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("scrapier/web-traffic-generator").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ "https://bebity.io" ], "proxyConfiguration": { "useApifyProxy": true } }' | apify call scrapier/web-traffic-generator --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapier/web-traffic-generator", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "πŸš€ Web Traffic Generator", "description": null, "version": "1.0", "x-build-id": "zl8cJ5LhzukXaIurM" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapier~web-traffic-generator/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapier-web-traffic-generator", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapier~web-traffic-generator/runs": { "post": { "operationId": "runs-sync-scrapier-web-traffic-generator", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapier~web-traffic-generator/run-sync": { "post": { "operationId": "run-sync-scrapier-web-traffic-generator", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "πŸ”— Website URLs", "minItems": 1, "type": "array", "description": "One or more website URLs to send visits to. Paste them in bulk β€” every visit picks one of these URLs in round-robin order. A URL without http(s):// is treated as https://.", "items": { "type": "string" } }, "numberOfVisitors": { "title": "#️⃣ Number of visits", "minimum": 1, "maximum": 100000, "type": "integer", "description": "Total number of browser visits to generate across all URLs.", "default": 10 }, "simultaneousVisitors": { "title": "πŸ§‘β€πŸ€β€πŸ§‘ Concurrent visits", "minimum": 1, "maximum": 100, "type": "integer", "description": "How many visits run in parallel. Higher values finish faster but need more memory β€” increase the run's RAM if you set this above 5.", "default": 5 }, "stayDurationSeconds": { "title": "βŒ› Time on page", "minimum": 1, "maximum": 3600, "type": "integer", "description": "How long each visit stays on a page, scrolling naturally, before moving on.", "default": 10 }, "useDesktop": { "title": "πŸ–₯️ Desktop devices", "type": "boolean", "description": "Emulate desktop browsers (Windows, macOS, Linux).", "default": true }, "useMobile": { "title": "πŸ“± Mobile devices", "type": "boolean", "description": "Emulate mobile phones (Pixel, iPhone, Galaxy).", "default": true }, "useTablet": { "title": "πŸ“Ÿ Tablet devices", "type": "boolean", "description": "Emulate tablets (iPad, Galaxy Tab).", "default": true }, "crawlWebsite": { "title": "πŸ•ΈοΈ Crawl internal pages", "type": "boolean", "description": "When enabled, every visit also follows internal links and browses additional pages of the site.", "default": true }, "crawlMaxPages": { "title": "πŸ“„ Max pages per visit", "minimum": 1, "maximum": 200, "type": "integer", "description": "Maximum number of extra pages a single crawling visit will browse.", "default": 20 }, "crawlSameDomainOnly": { "title": "πŸ”’ Stay on the same domain", "type": "boolean", "description": "Restrict crawling to pages on the start URL's domain. Disable to also follow external links.", "default": true }, "proxyConfiguration": { "title": "πŸ›‘οΈ Proxy configuration", "type": "object", "description": "Proxy used for the fallback tiers. Every visit begins with a direct connection; if a site blocks it, the actor escalates automatically β€” first to a datacenter proxy, then to a residential proxy β€” and sticks with the higher tier for the rest of the run.", "default": { "useApifyProxy": true } }, "maxRetries": { "title": "πŸ” Residential retry attempts", "minimum": 1, "maximum": 10, "type": "integer", "description": "How many times to retry a blocked visit on the residential proxy before giving up on it.", "default": 3 }, "retryBackoffSeconds": { "title": "⏳ Retry backoff", "minimum": 0, "maximum": 60, "type": "integer", "description": "Base wait time between residential retry attempts. The wait grows with each attempt.", "default": 2 }, "navigationTimeoutMs": { "title": "⏱️ Navigation timeout", "minimum": 5000, "maximum": 300000, "type": "integer", "description": "How long to wait for a page to load before treating the visit as failed. Residential proxies often need 30–60 seconds.", "default": 90000 }, "headless": { "title": "Headless browser", "type": "boolean", "description": "Run browsers in headless mode. Always keep enabled on the Apify platform.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```