# Bulk HTTP Request Runner: Call Any API or URL from a List (`ataism/http-runner`) Actor Run a list of HTTP requests or one template over many rows. Retries, JSON capture, per-host rate limits, robots.txt. $0.80 per 1,000. No proxies. - **URL**: https://apify.com/ataism/http-runner.md - **Developed by:** [Kaizoku Data](https://apify.com/ataism) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.80 / 1,000 request with a saved responses This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Bulk HTTP Request Runner: Call Any API or URL from a List Send a list of HTTP requests, or one request template over many rows of data, and get one dataset row per request: status, timing, response headers and the response body (JSON parsed). **$0.80 per 1,000 requests** (pay per event, any HTTP status you get back is a result). > **For APIs and sites you are allowed to call.** Your own services, APIs you have keys or permission for, public APIs whose terms allow automated use. Not for load testing, not for scraping sites that forbid it, not for getting around rate limits or blocks. This Actor is slow on purpose (1 request per second per host by default) and refuses to work around a refusal. See [Responsible use](#responsible-use). - **List or template.** Give URLs, full request objects (method, headers, body), or a template with `{{field}}` filled from your rows (for example 5,000 order ids against `https://api.example.com/orders/{{id}}`). - **Retries that respect the server.** Network errors, timeouts, 5xx and 429 are retried with growing waits that honour `Retry-After` (capped; a longer wait ends the request as a free row). - **Response capture.** JSON is parsed, text is kept, binary is skipped; up to 1 MB per response with a `truncated` flag; response headers as a short list, your own list, or all. - **Redirects, checked.** Up to 5, every hop re-checked (public addresses only, no platform hosts, robots.txt for GET and HEAD). - **Your secrets stay yours.** Header values you send (API keys, tokens) are never logged, and the request summary in each row lists header names only. A server can echo your headers back in its response body (some test endpoints do); the body is saved as the response, so do not call endpoints that echo secrets. ### Quick start 1. Open the Actor, keep the default input (3 GET requests to a public sandbox API) and press Start. It takes about 5 seconds. 2. Replace Requests with your own list, or use URLs plus the shared Method / Headers / Body. 3. Download the dataset as JSON, CSV or Excel, or call the Actor from code (see below). ### Input | Field | What it does | |---|---| | `requests` | List of URLs or objects `{url, method, headers, body, id}`. `body` is text or a JSON object/array. | | `urls` | Plain URL list that uses the shared method, headers and body. May be combined with `requests`. | | `rows` | Optional list of objects. Every request is sent once per row; `{{field}}` (dotted paths allowed) is replaced in the URL (percent-encoded), header values and body. Requests x rows at most 100,000. | | `method`, `headers`, `body` | Shared defaults for `urls` and for requests that do not set their own. A body that is JSON text is sent as `application/json`. Headers that claim a client address or change routing are refused (`Forwarded`, `X-Forwarded-*`, `X-Original-*`, `X-Real-IP`, anything with `client-ip`, `real-ip`, `user-ip` or `remote-addr` in its name, `X-Host`, method overrides, `Via`). | | `writeHosts` | Exact host names that may receive POST, PUT, PATCH and DELETE. Required for any write request; no wildcards. | | `requestsPerSecondPerHost` | Default 1, from 0.2 to 5 at 256 MB. The minimum rises with memory so one wait never costs more than 10% of the request price: 0.37 at 512 MB, 0.73 at 1,024 MB (a lower value is raised, with a warning in the log). Per registrable domain and per IP address (`a.example.com` and `b.example.com`, or two names for one server, share one budget). Counts retries, redirects and robots.txt. | | `timeoutSecs` | Default 30 (1-120). Starts after the rate-limit wait. | | `maxRetries`, `maxRetryAfterSecs`, `retryNonIdempotent` | Default 2 retries (max 3); longest `Retry-After` to wait (default and maximum 30 s); retry POST/PATCH after 5xx and network errors (off). | | `followRedirects`, `maxRedirects` | Default on, at most 5. | | `captureBody`, `maxBodyKb` | `auto` (JSON parsed), `text` or `none`; per-response cap, default 256 KB, max 1,024 KB. | | `responseHeaders`, `responseHeaderNames` | `common` (default), `all` (max 50) or `none`; or an explicit list. Sensitive names are `[redacted]`. | | `maxConcurrency` | Requests in flight, at most 10. | | `allowSlowHosts` | Honour a robots.txt Crawl-delay above 3 s (up to 5.5 s) instead of skipping the host (see Pricing). | | `userAgentSuffix` | Text appended to the Actor's own User-Agent. The User-Agent header itself cannot be set. | | `maxDurationMinutes` | Default 240. The run stops cleanly and keeps its results. | #### Example 1: a list of GET requests ```json { "requests": [ { "url": "https://quotes.toscrape.com/api/quotes?page=1" }, { "url": "https://quotes.toscrape.com/api/quotes?page=2", "id": "second" } ], "responseHeaders": "common" } ``` #### Example 2: one template over rows, with your own API key ```json { "urls": ["https://api.example.com/v1/orders/{{orderId}}"], "headers": { "Authorization": "Bearer YOUR_OWN_KEY", "Accept": "application/json" }, "rows": [{ "orderId": 1001 }, { "orderId": 1002 }], "userAgentSuffix": "acme-sync" } ``` #### Example 3: POST JSON to your own API ```json { "urls": ["https://api.example.com/v1/contacts"], "method": "POST", "writeHosts": ["api.example.com"], "headers": { "X-Api-Key": "YOUR_OWN_KEY" }, "body": { "name": "{{name}}", "plan": "{{plan}}" }, "rows": [{ "name": "Ann", "plan": "basic" }] } ``` ### Output One row per request (in completion order; `index` gives the input position): ```json { "index": 0, "request": { "method": "GET", "url": "https://quotes.toscrape.com/api/quotes?page=1", "headerNames": [], "bodyBytes": 0 }, "status": 200, "statusText": "OK", "ok": true, "attempts": 1, "durationMs": 804, "responseHeaders": { "content-type": "application/json", "date": "Tue, 20 Oct 2026 14:29:50 GMT" }, "body": { "has_next": true, "page": 1, "quotes": ["..."] }, "bodyType": "json", "bodyBytes": 2860, "truncated": false, "charged": true, "requestedAt": "2026-10-20T14:29:50.000Z" } ``` `request` lists header **names** only, never values and never the body. Values of sensitive query parameters (`key`, `token`, `secret`, `signature`, `api_key`, ...) are shown as `[redacted]` (the real request used the real value). A `RUN_SUMMARY` record in the key-value store has counts, events charged, bytes, failures by code, halted hosts and why the run stopped. ### Error codes (free rows) | `errorCode` | Meaning | |---|---| | `refused` | The server answered 401, 403 or 429 (or asked to wait longer than `maxRetryAfterSecs`). Status and body are recorded; not charged; never retried with another method. | | `challenge` | A challenge or CAPTCHA page. Recorded, not charged, never solved. | | `host-halted` | The host (registrable domain) refused 3 requests in this run, so it is not contacted again. Its queued requests are answered at once. | | `host-cap` | At most 10,000 requests per host (registrable domain) per run (retries, redirect hops and robots.txt count). | | `blocked-by-robots`, `robots-unreachable` | robots.txt disallows the path for GET/HEAD, or could not be read (5xx, network) after one retry. | | `slow-host` | robots.txt asks for a Crawl-delay above 3 s and `allowSlowHosts` is off, or above 5.5 s (always). Both limits shrink with memory: multiply by 256 and divide by the run's MB (512 MB: 1.5 s and 2.75 s). | | `write-host-not-allowed` | A POST/PUT/PATCH/DELETE to a host that is not in `writeHosts`. Nothing was sent. | | `redirect-blocked`, `too-many-redirects` | A write request was redirected to another origin; or more redirects than allowed. | | `platform` | Social-media, Google and similar platform hosts are refused. | | `blocked` | Private, loopback or link-local address (also after a redirect), or a port that is not a web port. | | `invalid-request`, `invalid-url` | A bad template, header or URL. | | `dns`, `timeout`, `connection-reset`, `connection-refused`, `tls`, `network` | The request failed. | Invalid input (for example a `User-Agent` header, a wildcard in `writeHosts`, more than 100,000 requests) fails the run before anything is sent. ### Pricing | Event | Price | When | |---|---|---| | `request` | **$0.0008** | One per request that has a response saved in the dataset. | Plus Apify's standard start event. Not charged: network errors, timeouts, refusals (401, 403, 429, challenge pages), robots.txt blocks and every guardrail refusal above. A 404 or 500 is a result and is charged. | Requests | Cost | |---|---| | 100 | $0.08 | | 1,000 | $0.80 | | 10,000 | $8.00 | | 100,000 (run limit) | $80.00 | Speed is limited by politeness: one host at the default 1 request/s takes about 17 minutes per 1,000 requests, 5 requests/s about 3.5 minutes (the slowest rate allowed is 0.2 requests/s at 256 MB, 0.37 at 512 MB and 0.73 at 1,024 MB, so waiting stays within 10% of the price). Different hosts run in parallel (10 in flight), so a mixed list is much faster. Waiting is billed as run time by Apify; at the default 256 MB it is about 2% of the request price. A robots.txt Crawl-delay of 5.5 s is the longest we wait (it keeps one host at 10% of the price); above 3 s a host is skipped unless you turn on `allowSlowHosts`. The delay is honoured in full or the host is skipped, never shortened. ### Limits - 100,000 requests per run, 10,000 per host, 10 in flight, 5 requests/s per host at most (a host is its registrable domain and also its IP address, so subdomains and aliases of one server share one budget). - 1 MB request body, 1 MB stored per response, 50 headers. - Web ports only (80, 443, 3000, 4000, 5000, 8000, 8008, 8080, 8081, 8443, 8888, 9000). - No cookies are kept between requests. HTTP/1.1 only. No proxies, no browser, no JavaScript. - Above 20,000 requests the run needs at least 512 MB of memory; with less, the input is refused before anything is sent. - If retries, waits and failed requests use more than 3 s per charged row plus 300 s, the run stops (`stoppedBy: unpaid-work-cap`) and keeps its results. - If a run is restarted by the platform, requests that had not finished may be sent again (finished ones never are, and are never charged twice). Keep this in mind for POST requests. ### Responsible use You are responsible for having the right to call every URL you list. Use it for: - your own services and APIs; - APIs you have a key or permission for, within their rate limits; - public APIs whose terms allow automated use. Do not use it for load or stress testing, for scraping sites that forbid it, or for getting around limits, blocks or authentication. The Actor enforces this where it can: - **Public internet only.** Private, loopback, link-local, CGNAT, multicast and reserved addresses (IPv4 and every IPv6 form that embeds them) are refused, at connect time and after every redirect. - **Slow by design.** At most 1 request/s per host by default (5 maximum), 10 in flight, 10,000 requests per host and 100,000 per run. - **robots.txt.** GET and HEAD requests honour robots.txt (exact `http-runner-actor` token, longest match) and `Crawl-delay`. This Actor does not crawl, but it applies the same rule as our crawlers. POST, PUT, PATCH and DELETE are actions you ask for explicitly: they are not checked against robots.txt, but they only go to hosts you list in `writeHosts`, are never redirected to another host, and use the same rate limits. - **Refusals are final.** 401, 403, 429 and challenge pages are recorded and free; after 3 refusals from a host it is not contacted again. No retry with another method, no challenge solving, no proxies, no header rotation, no fingerprint tricks. - **Honest identity.** The User-Agent is always `http-runner-actor/0.1 (https://apify.com/ataism/http-runner)`; you may append a suffix but not replace it, and a User-Agent header in your input is rejected. - **Your credentials only.** Headers are sent only because you supplied them, are dropped on a redirect to another origin, and are never logged or saved. Nothing is sent that you did not give; cookies from responses are not kept. - **No platform hosts.** YouTube, TikTok, Instagram, Facebook, X, Vimeo, SoundCloud, Spotify, Twitch, Reddit, LinkedIn and Google domains are refused. Responses may contain personal data from the services you call. This Actor stores what they return in your dataset and nothing else; you decide what to call and how to handle the results. ### Use it via API, n8n or Make cURL: ```bash curl -X POST "https://api.apify.com/v2/acts/ataism~http-runner/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"urls": ["https://quotes.toscrape.com/api/quotes?page=1"]}' ``` JavaScript (`apify-client`): ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('ataism/http-runner').call({ urls: ['https://quotes.toscrape.com/api/quotes?page=1'] }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); ``` Python (`apify-client`): ```python from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("ataism/http-runner").call(run_input={"urls": ["https://quotes.toscrape.com/api/quotes?page=1"]}) items = client.dataset(run["defaultDatasetId"]).list_items().items ``` n8n: add the Apify node, operation **Run an Actor and get dataset**, Actor `ataism/http-runner`, and paste the input JSON. Make: use **Run an Actor** (Actor `ataism/http-runner`), then **Get Dataset Items** with the run's default dataset. # Actor input Schema ## `requests` (type: `array`): A list of requests. Each item is a URL string or an object with "url" and optionally "method" (GET, HEAD, POST, PUT, PATCH, DELETE), "headers" (object), "body" (text, or a JSON object/array) and "id" (copied to the result row). Items can use {{field}} placeholders filled from the Rows below. POST, PUT, PATCH and DELETE are only sent to hosts you list in Write hosts. At most 100,000 requests per run. ## `urls` (type: `array`): A plain list of URLs that all use the shared Method, Headers and Body below. URLs can contain {{field}} placeholders filled from Rows (values are percent-encoded in the URL). You can use this together with Requests. ## `rows` (type: `array`): Optional list of objects, for example \[{"id": 1, "name": "Ann"}]. Every request in Requests and URLs is sent once per row, with {{id}}, {{name}} (and dotted paths like {{user.id}}) replaced in the URL, header values and body. A missing field makes that request a free error row. Requests times rows must not exceed 100,000. ## `method` (type: `string`): Method for URLs and for requests that do not set their own. POST, PUT, PATCH and DELETE need Write hosts. ## `headers` (type: `object`): Headers sent with every request, for example your own API key: {"Authorization": "Bearer ..."}. Values are never logged and never saved in the results (rows list header names only). The User-Agent header cannot be set: the Actor always sends its own honest User-Agent (use User agent suffix to add text). Host, Content-Length and proxy headers are not allowed. Headers are dropped when a redirect leads to another origin. Method-override, client-address (Forwarded, X-Forwarded-*, True-Client-IP, anything with client-ip, real-ip, user-ip or remote-addr in its name, ...) and routing headers (Via, X-Host, X-Http-Host, X-Override-URL, X-Original-*, X-Envoy-\*, ...) are not allowed, and header values may only contain characters up to U+00FF. A server may echo headers back in its response body, which is saved as the response. ## `body` (type: `string`): Body for POST, PUT, PATCH and DELETE requests that do not set their own. Text that is a JSON object or array is sent as application/json (set a Content-Type header to override). {{field}} is filled from Rows. At most 1 MB. GET and HEAD cannot have a body. ## `writeHosts` (type: `array`): Exact host names that may receive POST, PUT, PATCH and DELETE, for example api.example.com. No wildcards. A write request to any other host is not sent (free write-host-not-allowed row). Only list hosts you are allowed to write to. Write requests are never redirected to another host. ## `requestsPerSecondPerHost` (type: `number`): How fast one host is called. Default 1, from 0.2 to 5 at 256 MB. The minimum rises with the run's memory so one wait never costs more than 10% of the request price: 0.37 at 512 MB, 0.73 at 1,024 MB (a lower value is raised, with a warning in the log). The limit applies to the registrable domain and to the IP address, so a.example.com and b.example.com, or two names for one server, share it. Retries, redirects and robots.txt count. A robots.txt Crawl-delay slows a host further and is never shortened. There is also a cap of 10,000 requests per host (registrable domain) per run and at most 10 requests in flight overall. ## `timeoutSecs` (type: `integer`): Per request, including redirects. Time spent waiting for the rate limit or a Crawl-delay is not counted. A timeout is a free error row. ## `maxRetries` (type: `integer`): Retries (at most 3) after a network error, timeout, 5xx or 429, with growing waits that honour Retry-After. GET, HEAD, PUT and DELETE are retried on all of these; POST and PATCH only on 429 (unless you turn on Retry unsafe methods). Refusals (401, 403, challenge pages) are never retried. ## `maxRetryAfterSecs` (type: `integer`): If a server asks to wait longer than this, the request is not retried and its row is free (status 429, error code refused). ## `retryNonIdempotent` (type: `boolean`): Also retry POST and PATCH after timeouts, network errors and 5xx. Only turn this on if sending the same request twice is harmless. ## `followRedirects` (type: `boolean`): Follow 301, 302, 303, 307 and 308 (every hop is checked again: public address only, platform hosts, robots.txt for GET and HEAD). Write requests are followed only within the same origin. ## `maxRedirects` (type: `integer`): At most 5. ## `captureBody` (type: `string`): auto: JSON responses are parsed into the body field, other text is kept as text. text: always text. none: status and headers only. Binary responses are not saved (bodyOmitted: binary). ## `maxBodyKb` (type: `integer`): Per response, default 256, maximum 1,024. Longer bodies are cut and the row says truncated: true. ## `responseHeaders` (type: `string`): common: a short list (content-type, content-length, cache-control, etag, location, retry-after, rate-limit headers, ...). all: every header, at most 50, values cut at 2,000 characters. none: no headers. Headers with names like set-cookie, authorization, token or secret are saved as \[redacted]. ## `responseHeaderNames` (type: `array`): Optional list of header names. When set, only these are saved (redaction still applies). ## `maxConcurrency` (type: `integer`): Across all hosts. At most 10. ## `allowSlowHosts` (type: `boolean`): Off: GET and HEAD requests to a host whose robots.txt asks for a Crawl-delay above 3 seconds are skipped (free slow-host rows), because waiting makes the run slow and its run-time cost approaches the request price. On: delays up to 5.5 seconds are honoured in full (never shortened) and the waiting is billed as run time. Hosts asking for more than 5.5 seconds are always skipped. Both limits shrink with memory (multiply by 256, divide by the run's MB). ## `userAgentSuffix` (type: `string`): Text appended after the Actor's own User-Agent (http-runner-actor/0.1 (https://apify.com/ataism/http-runner)), for example the name of your project, so a server operator can tell who is calling. The Actor's own User-Agent cannot be replaced. Do not put personal data here. ## `maxDurationMinutes` (type: `integer`): The run stops cleanly at this time and keeps the results so far. Default 240. ## `mode` (type: `string`): Internal: "selfTest" runs the maintainer health check and saves no items. ## Actor input object example ```json { "requests": [ { "url": "https://quotes.toscrape.com/api/quotes?page=1" }, { "url": "https://quotes.toscrape.com/api/quotes?page=2" }, { "url": "https://quotes.toscrape.com/api/quotes?page=3" } ], "method": "GET", "requestsPerSecondPerHost": 1, "timeoutSecs": 30, "maxRetries": 2, "maxRetryAfterSecs": 30, "retryNonIdempotent": false, "followRedirects": true, "maxRedirects": 5, "captureBody": "auto", "maxBodyKb": 256, "responseHeaders": "common", "maxConcurrency": 10, "allowSlowHosts": false, "maxDurationMinutes": 240, "mode": "run" } ``` # Actor output Schema ## `results` (type: `string`): One row per request: method, URL, status, timing, response headers and body, or the error code (JSON, CSV, Excel or XML). ## `runSummary` (type: `string`): Counts, events charged, bytes downloaded, failures by code, hosts that were stopped and why the run stopped. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "requests": [ { "url": "https://quotes.toscrape.com/api/quotes?page=1" }, { "url": "https://quotes.toscrape.com/api/quotes?page=2" }, { "url": "https://quotes.toscrape.com/api/quotes?page=3" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("ataism/http-runner").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "requests": [ { "url": "https://quotes.toscrape.com/api/quotes?page=1" }, { "url": "https://quotes.toscrape.com/api/quotes?page=2" }, { "url": "https://quotes.toscrape.com/api/quotes?page=3" }, ] } # Run the Actor and wait for it to finish run = client.actor("ataism/http-runner").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "requests": [ { "url": "https://quotes.toscrape.com/api/quotes?page=1" }, { "url": "https://quotes.toscrape.com/api/quotes?page=2" }, { "url": "https://quotes.toscrape.com/api/quotes?page=3" } ] }' | apify call ataism/http-runner --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,ataism/http-runner" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/alhsb6rgmWqLPhsh4/builds/jJYzUjsF5s2orcwiv/openapi.json