# HTTP Request Runner: Bulk API Calls, Retries, Proxies (`data-done-right/http-request-runner`) Actor

HTTP request runner and API request runner: send bulk API calls (GET, POST, PUT, PATCH, DELETE, HEAD) from a list, in parallel, with retries and backoff, optional proxy, per-request headers, body, auth and JSON picking. One dataset row per HTTP request with status, timing and body.

- **URL**: https://apify.com/data-done-right/http-request-runner.md
- **Developed by:** [Data Done Right](https://apify.com/data-done-right) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 requests

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## HTTP Request Runner: Bulk API Calls, Retries, Proxies

This **HTTP request runner** sends a list of API calls for you: **bulk API calls** with GET, POST, PUT, PATCH, DELETE or HEAD, run in parallel, with automatic **retries and backoff**, optional **proxies**, per-request headers, body and auth, and **JSON field picking**. You get one dataset row per request with the status code, timing, attempts and the response body. Use it as an **API request runner** for webhooks, data syncs, health checks, API testing and AI agents that need to call an API and get structured results back.

### What can you do with an API request runner

- **Bulk API calls from a list.** Paste hundreds or thousands of requests, or just a list of URLs for bulk GETs. Up to 50 run at the same time.
- **Retries with exponential backoff.** Network errors, timeouts and 429/500/502/503/504 are retried, and a `Retry-After` header is honored. POST and PATCH are retried only when that is safe (connection errors, 429, 503), so a request is not applied twice.
- **Any method, header, body and auth.** JSON, form or raw bodies, query `params`, bearer tokens, basic auth and API-key headers per request.
- **Secret headers.** Put API keys in `secretHeaders`. They are stored encrypted by Apify, sent with every request, and never logged or written to the dataset (even if the API echoes them back).
- **Pick fields with JSON paths.** Return only what you need, for example `{"title": "$.title", "ids": "$.items[*].id"}`.
- **Proxies.** Route calls through Apify Proxy or your own proxies; each retry uses a new IP.
- **Safe by default.** Requests to localhost, private networks, link-local and cloud-metadata addresses are blocked, including through DNS tricks and redirects. Redirects are followed manually (up to a limit you set) and re-checked at every hop. Credentials are dropped when a redirect leaves the original host.
- **Never crashes on a bad URL.** A failing request becomes a row with an `error`, and the rest of the batch continues.

### How to send bulk API calls

1. Add your calls to **Requests** (a JSON array), or paste URLs into **Or a list of URLs (bulk GET)**.
2. Put tokens in **Secret headers**, for example `{"Authorization": "Bearer YOUR_TOKEN"}`.
3. Optionally set **Pick fields from JSON responses**, **Retries**, **Concurrency** and **Proxy**.
4. Run it and download the rows as JSON, CSV or Excel, or read them through the Apify API.

### Input example

```json
{
  "requests": [
    {"url": "https://jsonplaceholder.typicode.com/todos/1", "pick": {"title": "$.title", "completed": "$.completed"}},
    {"url": "https://api.example.com/items", "method": "POST", "json": {"name": "demo"}, "id": "create-demo"},
    {"url": "https://api.example.com/search", "params": {"q": "apify", "limit": 5}, "auth": {"bearer": "token"}}
  ],
  "urls": ["https://jsonplaceholder.typicode.com/users/1"],
  "secretHeaders": {"X-Api-Key": "YOUR_KEY"},
  "retries": 3,
  "concurrency": 10
}
```

Each request accepts `url`, `method`, `headers`, `params`, `json` / `body` / `form`, `auth`, `pick` and your own `id`. A plain URL string also works as a request.

### Output

```json
{
  "index": 0,
  "url": "https://jsonplaceholder.typicode.com/todos/1",
  "method": "GET", "statusCode": 200, "ok": true, "durationMs": 374, "attempts": 1,
  "finalUrl": "https://jsonplaceholder.typicode.com/todos/1", "redirects": 0,
  "contentType": "application/json", "bodyType": "json", "bodySize": 83, "bodyTruncated": false,
  "body": {"userId": 1, "id": 1, "title": "delectus aut autem", "completed": false},
  "picked": {"title": "delectus aut autem", "completed": false},
  "error": null
}
```

- `ok` is true for a 2xx response. `error` is filled for HTTP errors (`HTTP 404`), timeouts, network errors, blocked destinations and invalid input.
- `body` is parsed JSON when the response is JSON, otherwise text. Binary responses (images, PDFs, archives) return `bodyType: "binary"` without the content. Bodies larger than **Max response body size** are cut and marked `bodyTruncated`.
- Turn on **Include response headers** to add `responseHeaders` to each row.
- A **SUMMARY** record in the key-value store counts ok and failed requests and the status codes.

### JSON path syntax for picking fields

`$.a.b`, `$.items[0]`, `$.items[-1]`, `$.items[*].id`, `$.data.*`, `$.items[1:3]` and `$..name` (any depth). Missing fields return `null`. Picking works on JSON responses only.

### HTTP request API for AI agents and workflows

Call this Actor from the Apify API, Python, JavaScript, Make, Zapier or n8n and use it as a managed HTTP client: your workflow sends a batch of requests and gets one clean result per request, with retries, proxy rotation and safe-network checks handled for you.

### Use it in Claude, ChatGPT and Cursor

Connect Apify's MCP server (mcp.apify.com) and your AI assistant can call APIs and read the results itself. For example: "call these 50 endpoints, retry failures, and give me the status and the `version` field from each." The main input is `requests`, with `urls` as a shortcut for bulk GETs.

### Use cases

- **API testing and monitoring:** hit endpoints on a schedule and check status codes and response times.
- **Bulk data fetching:** call a REST API for thousands of IDs without writing a loop.
- **Webhooks and syncs:** POST records to your system with retries.
- **Link and status checks:** HEAD a list of URLs and collect status codes.
- **AI agents:** a safe way to let an agent make HTTP calls.

### How much does it cost

Pay per event: **$0.0025 per request** (`request`). You are charged once per request, not per retry, and invalid requests (bad input, or a URL written directly as a private or internal address) are free. A hostname that only resolves to a private address, or a redirect into one, is blocked but still counts as a request. 1,000 requests cost about $2.50. Apify Silver plans get 10% off, Gold and higher 20%.

### FAQ

#### How do I send many API requests at once?

Put them in **Requests** (or paste URLs in the URLs list) and set **Concurrency** (default 10, up to 50). The Actor runs them in parallel and returns one row per request, with an `index` that matches your input position.

#### Is this a Postman or curl-loop alternative?

Yes, for bulk work. Instead of a Postman collection runner or a shell loop around curl, you send a list of requests and get one result row per request, with retries, backoff, optional proxies and picked JSON fields included. You pay per request with no start fee, and there is nothing to host.

#### Are retries charged?

No. The price is per request. Retries, backoff waits and redirects are included.

#### Are my API keys safe?

Put them in **Secret headers**. Apify stores that field encrypted, and the Actor never writes it to the log or the dataset. If an API echoes your key in its response, the value is replaced by `***`.

#### Can it call localhost or internal services?

No. Private networks, localhost, link-local and cloud-metadata addresses are blocked, also after DNS resolution and on every redirect. This keeps the Actor from being used to reach internal infrastructure.

#### Does it retry POST requests?

Only when it is safe: connection errors, 429 and 503. Turn on **Retry POST and PATCH on any failure** if your API is idempotent.

#### What happens when a request fails?

It becomes a row with `ok: false` and an `error`. The run continues and still succeeds.

#### Can it handle large responses?

Responses are cut at **Max response body size** (1 MB by default, up to 4 MB) and marked `bodyTruncated`. Binary content is not returned.

#### Do I need a proxy?

Usually not for APIs. Turn on **Proxy** when a site blocks datacenter traffic or rate-limits by IP.

### Related tools

- **Google Sheets Import & Export: Apify Data, No Login**: send results to a Google Sheet.
- **Slack Notifications: Send Data to Slack, Discord & Email**: alert your team when a check fails.

# Actor input Schema

## `requests` (type: `array`):

The HTTP requests to run, as a JSON array. Each item has "url" (required) and optional "method" (GET, POST, PUT, PATCH, DELETE or HEAD), "headers", "params" (query string), "json" or "body" or "form" (request body), "auth" ({"bearer": "token"} or {"username": "u", "password": "p"} or {"apiKey": "k", "headerName": "X-API-Key"}), "pick" and "id". Example: \[{"url": "https://jsonplaceholder.typicode.com/todos/1", "pick": {"title": "$.title"}}, {"url": "https://api.example.com/items", "method": "POST", "json": {"name": "demo"}}]. A plain URL string also works as an item.

## `urls` (type: `array`):

Quick mode for bulk calls: one URL per line, all sent with the default method, headers and pick below. Added after the Requests list. Example: https://api.example.com/items/1, https://api.example.com/items/2.

## `method` (type: `string`):

Method used for the URLs list and for requests that do not set their own.

## `headers` (type: `object`):

Headers added to every request (a request's own headers win). Not secret, so do not put tokens here. Example: {"Accept": "application/json"}.

## `secretHeaders` (type: `object`):

Headers sent with every request that are stored encrypted and never logged or written to the dataset. They override other headers of the same name. Example: {"Authorization": "Bearer YOUR_TOKEN"}.

## `pick` (type: `object`):

Fields to extract from JSON responses into the picked column, as {name: JSONPath}. Supports $.a.b, \[0], \[-1], \[*], .*, \[1:3] and ..key. Example: {"id": "$.data.id", "names": "$.items\[\*].name"}. A request's own pick wins.

## `includeResponseHeaders` (type: `boolean`):

Add the response headers of every request to its row.

## `retries` (type: `integer`):

Extra attempts after a network error, timeout or a retryable status. You pay once per request, not per retry.

## `backoffSecs` (type: `integer`):

First wait before a retry; doubles each time (max 30 s). A Retry-After header from the server is honored.

## `retryStatusCodes` (type: `array`):

HTTP statuses that trigger a retry. Default: 429, 500, 502, 503, 504.

## `retryUnsafeMethods` (type: `boolean`):

By default POST and PATCH are retried only on connection errors, 429 and 503, so a request is not applied twice. Turn on only if your API is idempotent.

## `followRedirects` (type: `boolean`):

Follow 301/302/303/307/308 redirects. Every hop is checked against the private-network block.

## `maxRedirects` (type: `integer`):

Redirect limit per request.

## `maxRequests` (type: `integer`):

Hard cap on the number of requests run. Extra ones are skipped.

## `maxBodyBytes` (type: `integer`):

Larger responses are cut at this size and marked bodyTruncated. JSON bodies are only parsed when complete.

## `timeoutSecs` (type: `integer`):

Time allowed for one attempt, redirects included.

## `concurrency` (type: `integer`):

Requests in flight at once (1-50).

## `proxyConfiguration` (type: `object`):

Send requests through Apify Proxy or your own proxies. Each retry uses a new session (new IP). Usually not needed for APIs.

## Actor input object example

```json
{
  "requests": [
    {
      "url": "https://jsonplaceholder.typicode.com/todos/1",
      "method": "GET",
      "pick": {
        "title": "$.title",
        "completed": "$.completed"
      }
    },
    {
      "url": "https://jsonplaceholder.typicode.com/posts/1",
      "method": "GET",
      "pick": {
        "title": "$.title"
      }
    }
  ],
  "method": "GET",
  "includeResponseHeaders": false,
  "retries": 2,
  "backoffSecs": 1,
  "retryStatusCodes": [
    429,
    500,
    502,
    503,
    504
  ],
  "retryUnsafeMethods": false,
  "followRedirects": true,
  "maxRedirects": 5,
  "maxRequests": 1000,
  "maxBodyBytes": 1000000,
  "timeoutSecs": 30,
  "concurrency": 10,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

One item per request: status code, timing, attempts and the response body (all result items of this run, as JSON, CSV, Excel and more).

## `records` (type: `string`):

Key-value store records: the run summary (SUMMARY: ok/failed counts and status codes) and notes on any problems.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "requests": [
        {
            "url": "https://jsonplaceholder.typicode.com/todos/1",
            "method": "GET",
            "pick": {
                "title": "$.title",
                "completed": "$.completed"
            }
        },
        {
            "url": "https://jsonplaceholder.typicode.com/posts/1",
            "method": "GET",
            "pick": {
                "title": "$.title"
            }
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("data-done-right/http-request-runner").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "requests": [
        {
            "url": "https://jsonplaceholder.typicode.com/todos/1",
            "method": "GET",
            "pick": {
                "title": "$.title",
                "completed": "$.completed",
            },
        },
        {
            "url": "https://jsonplaceholder.typicode.com/posts/1",
            "method": "GET",
            "pick": { "title": "$.title" },
        },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("data-done-right/http-request-runner").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "requests": [
    {
      "url": "https://jsonplaceholder.typicode.com/todos/1",
      "method": "GET",
      "pick": {
        "title": "$.title",
        "completed": "$.completed"
      }
    },
    {
      "url": "https://jsonplaceholder.typicode.com/posts/1",
      "method": "GET",
      "pick": {
        "title": "$.title"
      }
    }
  ]
}' |
apify call data-done-right/http-request-runner --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,data-done-right/http-request-runner"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/k734NqC2kM0dYJACF/builds/MhhCH6op3gT2crWBz/openapi.json