# HTTP Request Runner: Bulk API Calls with Retries (`getascraper/http-request-runner`) Actor

Send up to 1000 HTTP requests in one batch with retries, redirect chains, and JSON parsing built in. Paste a curl command from DevTools or Postman instead of hand-building requests. Works with n8n, Make, Zapier, and MCP agents. Skip the retry and concurrency code. From $0.0006 per response.

- **URL**: https://apify.com/getascraper/http-request-runner.md
- **Developed by:** [GetAScraper](https://apify.com/getascraper) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 4 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.61 / 1,000 http responses

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🔌 HTTP Request Runner: Bulk API Calls with Retries

<table width="100%">
<tr>
<td style="padding:24px 28px;background:#E6F5F3;border:1px solid #B8E0DB;border-top:4px solid #0F766E;border-radius:12px">
<span style="font-size:23px;font-weight:800;color:#1C1917;line-height:1.3">Send hundreds of requests, get back clean data</span><br>
<span style="font-size:15px;color:#57534E;line-height:1.6">Batch HTTP requests with automatic retries, redirect tracking, and JSON parsing built in, so you don't have to write that layer yourself.</span>
</td>
</tr>
</table>

<table width="100%">
<tr>
<td style="padding:14px 12px;width:25%;background:#FFFFFF;border:1px solid #B8E0DB;border-radius:10px 0 0 10px;vertical-align:top">
<span style="font-size:15px;font-weight:800;color:#0B5A54">📋 Paste a curl command</span><br>
<span style="font-size:12px;color:#57534E">Skip building request objects by hand, copy one from DevTools or Postman and run it as is.</span>
</td>
<td style="padding:14px 12px;width:25%;background:#FFFFFF;border:1px solid #B8E0DB;border-left:none;vertical-align:top">
<span style="font-size:15px;font-weight:800;color:#0B5A54">🔀 Full redirect visibility</span><br>
<span style="font-size:12px;color:#57534E">See every hop in the chain, not just the final URL.</span>
</td>
<td style="padding:14px 12px;width:25%;background:#FFFFFF;border:1px solid #B8E0DB;border-left:none;vertical-align:top">
<span style="font-size:15px;font-weight:800;color:#0B5A54">🛡️ Works behind a proxy</span><br>
<span style="font-size:12px;color:#57534E">Route through Apify Proxy when a target rate-limits or geo-blocks by IP, optional and off by default.</span>
</td>
<td style="padding:14px 12px;width:25%;background:#FFFFFF;border:1px solid #B8E0DB;border-left:none;border-radius:0 10px 10px 0;vertical-align:top">
<span style="font-size:15px;font-weight:800;color:#0B5A54">📦 Nothing gets truncated</span><br>
<span style="font-size:12px;color:#57534E">Oversized and binary responses are saved in full instead of being cut off.</span>
</td>
</tr>
</table>

Point it at up to 1,000 URLs, or paste a single curl command, and get one dataset row per request: status, redirect chain, parsed JSON, timing, and any error. It runs retries, concurrency, and rate limiting for you, so your pipeline, monitor, or AI agent only has to read the results.

### 🎯 Common workflows

- **Pipeline builders (n8n, Make, Zapier):** Replace a chain of manual HTTP nodes with one batch call, then stream each result to your workflow with a callback webhook as it completes.
- **QA and uptime engineers:** Check hundreds of endpoints in one run and see instantly which ones failed, redirected, or didn't match the status code you expected.
- **AI agent and MCP developers:** Give an agent one tool that can call any API: batched, retried, and parsed into JSON it can use directly.
- **Data teams:** Pull from a list of API endpoints on a schedule and land clean, structured rows in a dataset without maintaining request code.

### 🚀 How to use it

<table width="100%">
<tr>
<td style="padding:16px 14px;width:33%;background:#E6F5F3;border:1px solid #B8E0DB;border-radius:10px 0 0 10px;vertical-align:top">
<span style="font-size:12px;font-weight:800;color:#0F766E;letter-spacing:1px">STEP 1</span><br>
<span style="font-size:14px;font-weight:700;color:#1C1917">Add your requests</span><br>
<span style="font-size:12px;color:#57534E">List URLs and options, or paste a curl command.</span>
</td>
<td style="padding:16px 14px;width:33%;background:#E6F5F3;border:1px solid #B8E0DB;border-left:none;vertical-align:top">
<span style="font-size:12px;font-weight:800;color:#0F766E;letter-spacing:1px">STEP 2</span><br>
<span style="font-size:14px;font-weight:700;color:#1C1917">Set the rules</span><br>
<span style="font-size:12px;color:#57534E">Concurrency, retries, auth, and proxy, all optional.</span>
</td>
<td style="padding:16px 14px;width:33%;background:#E6F5F3;border:1px solid #B8E0DB;border-left:none;border-radius:0 10px 10px 0;vertical-align:top">
<span style="font-size:12px;font-weight:800;color:#0F766E;letter-spacing:1px">STEP 3</span><br>
<span style="font-size:14px;font-weight:700;color:#1C1917">Get structured results</span><br>
<span style="font-size:12px;color:#57534E">Status, JSON, redirects, and timing for every request.</span>
</td>
</tr>
</table>

### 📥 Input

At least one item in `requests` or a `curlCommand` is required.

| Field | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| `requests` | array | Conditional | Request objects: `id`, `label`, `url`, `method`, `headers`, `query`, `body`, `expectedStatusCodes`. Only `url` is required per item. |
| `curlCommand` | string | Conditional | A single curl command to parse into a request, in addition to `requests`. |
| `maxConcurrency` | integer | No | Requests running at the same time. |
| `requestsPerSecond` | integer | No | Extra rate limit independent of concurrency. `0` disables it. |
| `timeoutSecs` | integer | No | Seconds to wait per attempt. |
| `maxRetries` | integer | No | Retry attempts after a network error or a retryable status code. |
| `retryOnStatusCodes` | array of integers | No | Status codes that trigger a retry. |
| `followRedirects` | boolean | No | Follow HTTP redirects automatically. |
| `maxRedirects` | integer | No | Maximum redirect hops per request. |
| `parseJson` | boolean | No | Parse JSON responses into a structured `json` field. |
| `maxResponseBytes` | integer | No | Response bodies larger than this are truncated inline. |
| `saveOversizedBodyToKeyValueStore` | boolean | No | Save oversized or binary bodies in full instead of truncating. |
| `failOnErrorStatus` | boolean | No | Also populate `errorMessage` for non-2xx responses. |
| `expectedStatusCodes` | array of integers | No | Default status-code assertion applied to every request without its own. |
| `authType` | enum | No | `none`, `bearer`, or `basic`, applied to requests without their own Authorization header. |
| `authBearerToken` | string (secret) | No | Used when `authType` is `bearer`. |
| `authBasicUsername` | string | No | Used when `authType` is `basic`. |
| `authBasicPassword` | string (secret) | No | Used when `authType` is `basic`. |
| `proxyConfiguration` | object | No | Optional Apify Proxy configuration. Off by default. |
| `callbackUrl` | string | No | POST each result to this URL as JSON the moment it completes. |

#### Curl import example

```json
{
    "curlCommand": "curl -X POST https://api.example.com/orders -H \"Authorization: Bearer sk_live_xxx\" -d '{\"orderId\":42}'",
    "maxConcurrency": 5
}
```

#### Batch example with an assertion

```json
{
    "requests": [
        { "id": "home", "url": "https://example.com", "expectedStatusCodes": [200] },
        { "id": "api-health", "url": "https://api.example.com/health", "expectedStatusCodes": [200] }
    ],
    "maxConcurrency": 10,
    "requestsPerSecond": 20
}
```

### 📤 Output

One row per request:

```json
{
    "requestId": "api-health",
    "label": "health check",
    "requestedUrl": "https://api.example.com/health",
    "finalUrl": "https://api.example.com/health",
    "method": "GET",
    "statusCode": 200,
    "statusText": "OK",
    "ok": true,
    "isBroken": false,
    "isRedirect": false,
    "redirectChain": [],
    "contentType": "application/json",
    "json": { "status": "ok", "uptimeSeconds": 481293 },
    "bodyBytes": 34,
    "bodyTruncated": false,
    "assertionPassed": true,
    "durationMs": 214,
    "attempts": 1,
    "fetchedAt": "2026-08-19T18:00:43.126Z"
}
```

Three dataset views are ready in the Output tab: **Responses** for status and timing, **Response Bodies** for full payloads and parsed JSON, and **Redirects & Errors** for chains and failures. A run summary with success rate, status code breakdown, and average response time is also saved to the key-value store.

#### Field reference

| Field | Type | Description |
| :--- | :--- | :--- |
| `requestId`, `label` | string | Your own identifiers, echoed back for correlation. |
| `requestedUrl`, `finalUrl` | string | The URL you sent and the URL after any redirects. |
| `method` | string | HTTP method used. |
| `statusCode`, `statusText` | integer, string | Response status. |
| `ok` | boolean | `true` for 2xx responses. |
| `isBroken` | boolean | `true` for 4xx, 5xx, or no response. |
| `isRedirect`, `redirectChain`, `redirectCount` | boolean, array, integer | Whether the request redirected, and every hop it took. |
| `contentType` | string | Response `Content-Type` header. |
| `body` | string | Raw response text, or base64 for binary content. |
| `json` | object | Parsed JSON body, when the response is JSON. |
| `bodyBytes`, `bodyTruncated`, `bodyStoredKey` | integer, boolean, string | Response size, whether it was cut off inline, and where the full body was saved if so. |
| `assertionPassed` | boolean | Whether the status code matched `expectedStatusCodes`, when set. |
| `durationMs`, `attempts` | integer | Timing and retry count for the request. |
| `errorType`, `errorMessage` | string | Failure details when no response was received. |
| `fetchedAt` | string | Timestamp the request completed. |

### 💰 Pricing

This Actor is pay per result. You are charged for requests that complete, and an empty or failed run costs nothing beyond the fixed start fee. There is no subscription.

### ⭐ Enjoying HTTP Request Runner?

<table width="100%">
<tr>
<td style="padding:20px 24px 14px;background:#E6F5F3;border:1px solid #B8E0DB;border-left:5px solid #0F766E;border-radius:10px 10px 0 0">
<span style="font-size:20px;letter-spacing:4px">⭐ ⭐ ⭐ ⭐ ⭐</span><br>
<span style="font-size:17px;font-weight:800;color:#1C1917">If this Actor saved you from writing retry and concurrency code by hand, leave a rating.</span><br>
<span style="font-size:14px;color:#57534E">A 5-star rating takes 10 seconds and helps other pipeline builders and agent developers find it. Your feedback also tells us what to build next.</span>
</td>
</tr>
<tr>
<td style="padding:0;background:#0F766E;border:1px solid #B8E0DB;border-top:none;border-radius:0 0 10px 10px;text-align:center">
<a href="https://apify.com/getascraper/http-request-runner/reviews" style="display:block;padding:13px 16px;color:#FFFFFF;text-decoration:none;font-weight:800;font-size:15px;letter-spacing:0.3px">★&nbsp;&nbsp;Rate this Actor on Apify</a>
</td>
</tr>
</table>

### 🛡️ Tips and limits

- Requests run up to `maxConcurrency` at a time. Add `requestsPerSecond` on top of that when a target rejects bursts even at low concurrency.
- `retryOnStatusCodes` defaults to codes worth retrying (408, 425, 429, 500, 502, 503, 504). A 4xx code outside that list means the request itself is wrong and will fail identically on retry, so it is not retried.
- Turn on `saveOversizedBodyToKeyValueStore` to keep full binary or large responses without bloating the dataset.
- `proxyConfiguration` is optional and off by default. Turn it on only for targets that rate-limit or geo-block by IP.

### ❓ FAQ

#### Can I import a request straight from my browser or Postman?

Yes. Copy the curl command and paste it into `curlCommand`. It runs alongside anything in `requests`.

#### Does it support proxies?

Yes, optionally through `proxyConfiguration`. It is off by default since most API calls should go direct.

#### What happens when a request fails?

It retries according to `maxRetries` and `retryOnStatusCodes`, then the row still gets pushed with `isBroken: true` and an `errorMessage`, so failures stay visible instead of disappearing from the dataset.

#### Can I stream results to my own system as they complete?

Yes. Set `callbackUrl` and each result is POSTed there as JSON the moment it finishes.

#### Does it run as a standby web server?

No. Each run processes its batch, writes results and a run summary, then exits.

### 🔗 Other actors

- [Google Lens OCR API](https://apify.com/getascraper/google-lens-ocr) ↗ - Batch image to text extraction with bounding boxes, for document and RAG pipelines.
- [arXiv scraper for RAG](https://apify.com/getascraper/arxiv-rag-extractor) ↗ - Research papers as chunked JSON for retrieval pipelines.
- [Domain Health Monitor](https://apify.com/getascraper/domain-health-monitor) ↗ - WHOIS, DNS, and SSL monitoring for a list of domains.
- [Bulk Image Downloader](https://apify.com/getascraper/bulk-image-downloader) ↗ - Download image batches with metadata, hashes, and Apify storage output.

# Actor input Schema

## `requests` (type: `array`):

Array of request objects: { id, label, url, method, headers, query, body, expectedStatusCodes }. Only url is required per item.

## `curlCommand` (type: `string`):

Paste a single curl command (copied from browser DevTools, Postman, or a terminal) and it will be parsed into a request. Runs in addition to any requests above.

## `maxConcurrency` (type: `integer`):

Maximum number of requests running at the same time.

## `requestsPerSecond` (type: `integer`):

Independent rate limit separate from concurrency, useful for APIs that reject bursts even under a low concurrency limit. 0 means no extra throttle.

## `timeoutSecs` (type: `integer`):

Seconds to wait for a response before treating the attempt as failed.

## `maxRetries` (type: `integer`):

Retry attempts after a network error or a retryable status code, using exponential backoff with jitter.

## `retryOnStatusCodes` (type: `array`):

Response status codes that trigger a retry. A 4xx code other than 408/425/429 means your request is malformed and will never succeed on retry, so it is not retried by default.

## `followRedirects` (type: `boolean`):

Automatically follow HTTP redirects (301, 302, 307, 308).

## `maxRedirects` (type: `integer`):

Maximum redirect hops to follow per request.

## `parseJson` (type: `boolean`):

When the response Content-Type is JSON, parse it into a structured json field instead of leaving it as raw text.

## `maxResponseBytes` (type: `integer`):

Response bodies larger than this are truncated in the body field. Combine with Save Oversized Bodies below to keep the full content without bloating the dataset.

## `saveOversizedBodyToKeyValueStore` (type: `boolean`):

When a body exceeds Max Response Bytes or is binary (images, PDFs, archives), store the full content in the key-value store and return a link in bodyStoredKey instead of silently truncating it.

## `failOnErrorStatus` (type: `boolean`):

When enabled, a non-2xx response also populates the errorMessage field, on top of the normal statusCode/ok fields.

## `expectedStatusCodes` (type: `array`):

Optional. If set, every response is checked against this list and the result includes assertionPassed, useful for uptime or contract checks. Leave empty to skip the assertion. A per-request expectedStatusCodes field overrides this default.

## `authType` (type: `string`):

Applied automatically to all requests that don't already set an Authorization header of their own.

## `authBearerToken` (type: `string`):

Used when Default Authentication is set to Bearer token.

## `authBasicUsername` (type: `string`):

Used when Default Authentication is set to Basic auth.

## `authBasicPassword` (type: `string`):

Used when Default Authentication is set to Basic auth.

## `proxyConfiguration` (type: `object`):

Optional. Off by default since most API calls should go direct; turn it on for targets that rate-limit or geo-block by IP.

## `callbackUrl` (type: `string`):

If set, each result row is POSTed to this URL as JSON the moment it completes, the same pattern tools like n8n and Clay use for streaming results.

## Actor input object example

```json
{
  "requests": [
    {
      "label": "example",
      "url": "https://httpbin.org/get",
      "method": "GET"
    }
  ],
  "curlCommand": "",
  "maxConcurrency": 5,
  "requestsPerSecond": 0,
  "timeoutSecs": 30,
  "maxRetries": 2,
  "retryOnStatusCodes": [
    408,
    425,
    429,
    500,
    502,
    503,
    504
  ],
  "followRedirects": true,
  "maxRedirects": 5,
  "parseJson": true,
  "maxResponseBytes": 1000000,
  "saveOversizedBodyToKeyValueStore": true,
  "failOnErrorStatus": false,
  "expectedStatusCodes": [],
  "authType": "none",
  "authBasicUsername": "",
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "callbackUrl": ""
}
```

# Actor output Schema

## `responses` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

## `storedBodies` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "requests": [
        {
            "label": "example",
            "url": "https://httpbin.org/get",
            "method": "GET"
        }
    ],
    "curlCommand": "",
    "maxConcurrency": 5,
    "requestsPerSecond": 0,
    "timeoutSecs": 30,
    "maxRetries": 2,
    "retryOnStatusCodes": [
        408,
        425,
        429,
        500,
        502,
        503,
        504
    ],
    "followRedirects": true,
    "maxRedirects": 5,
    "parseJson": true,
    "maxResponseBytes": 1000000,
    "saveOversizedBodyToKeyValueStore": true,
    "failOnErrorStatus": false,
    "expectedStatusCodes": [],
    "authType": "none",
    "authBasicUsername": "",
    "proxyConfiguration": {
        "useApifyProxy": false
    },
    "callbackUrl": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("getascraper/http-request-runner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "requests": [{
            "label": "example",
            "url": "https://httpbin.org/get",
            "method": "GET",
        }],
    "curlCommand": "",
    "maxConcurrency": 5,
    "requestsPerSecond": 0,
    "timeoutSecs": 30,
    "maxRetries": 2,
    "retryOnStatusCodes": [
        408,
        425,
        429,
        500,
        502,
        503,
        504,
    ],
    "followRedirects": True,
    "maxRedirects": 5,
    "parseJson": True,
    "maxResponseBytes": 1000000,
    "saveOversizedBodyToKeyValueStore": True,
    "failOnErrorStatus": False,
    "expectedStatusCodes": [],
    "authType": "none",
    "authBasicUsername": "",
    "proxyConfiguration": { "useApifyProxy": False },
    "callbackUrl": "",
}

# Run the Actor and wait for it to finish
run = client.actor("getascraper/http-request-runner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "requests": [
    {
      "label": "example",
      "url": "https://httpbin.org/get",
      "method": "GET"
    }
  ],
  "curlCommand": "",
  "maxConcurrency": 5,
  "requestsPerSecond": 0,
  "timeoutSecs": 30,
  "maxRetries": 2,
  "retryOnStatusCodes": [
    408,
    425,
    429,
    500,
    502,
    503,
    504
  ],
  "followRedirects": true,
  "maxRedirects": 5,
  "parseJson": true,
  "maxResponseBytes": 1000000,
  "saveOversizedBodyToKeyValueStore": true,
  "failOnErrorStatus": false,
  "expectedStatusCodes": [],
  "authType": "none",
  "authBasicUsername": "",
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "callbackUrl": ""
}' |
apify call getascraper/http-request-runner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,getascraper/http-request-runner"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/4XZRbY5NyS1teiQre/builds/ZfxxIxLEEDJU13tlS/openapi.json