# HTTP Request Tool for AI Agents (`tinplatelabs/agent-http`) Actor Send batches of HTTP requests and get agent-ready results: parsed JSON, clean Markdown from HTML, retries with Retry-After, redirect chain, timing, TLS expiry, SSRF protection and redacted secrets. URL health mode for link checks. Priced per response. - **URL**: https://apify.com/tinplatelabs/agent-http.md - **Developed by:** [tinplatelabs](https://apify.com/tinplatelabs) (community) - **Categories:** Developer tools, AI, Automation - **Stats:** 7 total users, 6 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $1.00 / 1,000 request completeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## HTTP Request Tool for AI Agents Give it a batch of HTTP requests and get back **one clean, agent-ready record per request**: status, headers, the body as **parsed JSON**, **Markdown of the page's main content** (for HTML) or text, the full **redirect chain**, a **timing breakdown** and **TLS certificate** details. Built-in retries, timeouts, size limits and **SSRF protection**. A **URL health** mode checks lists of links. Built for AI agents (Claude, ChatGPT, Cursor and others through the [Apify MCP server](https://mcp.apify.com)) that need to call APIs or read web pages without blowing up their context window, and for anyone who needs a reliable HTTP client in an Apify workflow. ### Why this one - **Bodies an agent can read.** JSON comes back parsed. HTML becomes Markdown of the main content, with navigation, cookie banners, sidebars and footers stripped and links made absolute. Binary files (images, PDFs, ZIPs) are saved to the key-value store and returned as a URL instead of a blob. - **Token budget.** `maxBodyTokens` (default 8,000) cuts long bodies at a paragraph boundary and says so (`truncated: true`, `bodyChars`, `bodyTokensEstimate`), so one huge page can't flood the context. - **Robust.** Retries on 429/5xx and connection failures with exponential backoff, honouring `Retry-After`. POST/PATCH are only retried when it's safe (429 or connection failure) unless you allow it. - **Transparent.** Every redirect hop, attempt and retry is listed; timing is split into DNS, connect, TLS, waiting (time to first byte) and download. - **Safe by default.** Private, loopback, link-local and cloud-metadata addresses (e.g. `169.254.169.254`) are refused, including after redirects and DNS resolution. `Authorization`, cookies and API-key headers and query parameters are redacted in the output and log, and credentials are not forwarded to another host on a redirect. No cookies are kept between requests unless you turn on `session`. - **Pay only for responses.** Any HTTP response is charged, including 404 or 500 (that's a successful request). DNS failures, timeouts, blocked addresses and invalid input cost nothing. ### Input ```json { "requests": [ {"method": "GET", "url": "https://httpbin.org/json"}, {"method": "POST", "url": "https://api.example.com/v1/search", "json": {"q": "widgets"}, "headers": {"X-Api-Key": "..."}, "label": "search"}, "https://example.com" ], "authorization": "Bearer sk-...", "maxBodyTokens": 4000 } ``` | Field | Default | What it does | |---|---|---| | `mode` | `request` | `request` returns full responses; `url-health` checks links without downloading bodies | | `requests` | | URL strings or objects: `method`, `url`, `headers`, `query`, one of `json` / `form` / `body`, `timeoutSecs`, `followRedirects`, `label` | | `urls` | | Plain URLs to GET (or check). Bare domains get `https://` | | `headers` | | Headers for every request (a request's own headers win) | | `authorization` | | Secret `Authorization` value for every request (stored encrypted, never output) | | `userAgent` | identifies this Actor | User-Agent header | | `timeoutSecs` | `30` | Time limit per attempt, redirects included (1–300) | | `maxRetries` | `2` | Retries on 429/500/502/503/504 and connection failures (0–5) | | `maxRetryWaitSecs` | `30` | Longest `Retry-After` to wait for | | `retryUnsafeMethods` | `false` | Also retry POST/PATCH on 5xx and timeouts | | `followRedirects` / `maxRedirects` | `true` / `10` | Redirect handling | | `htmlFormat` | `markdown` | `markdown` (main content), `markdown-full`, `text` or `html` | | `parseJson` | `true` | Return JSON parsed | | `maxBodyTokens` / `maxBodyChars` | `8000` / `0` | Body budget (≈4 characters per token); 0 = no limit; the smaller applies | | `maxResponseMb` | `10` | Download limit per body (1–50) | | `storeBinary` | `true` | Save binary bodies to the key-value store | | `session` | `false` | Keep cookies across the run's requests | | `verifyTls` | `true` | Reject invalid certificates | | `maxConcurrency` | `5` | Requests in flight (1–20) | | `healthMethod` / `tlsWarningDays` | `GET` / `14` | url-health mode options | ### Output One dataset item per request, in completion order (`index` is the position in the input). Abridged: ```json { "index": 0, "label": "docs", "method": "GET", "url": "https://example.com/docs", "status": "ok", "statusCode": 200, "statusText": "OK", "ok": true, "finalUrl": "https://www.example.com/docs/", "redirects": [{"url": "https://example.com/docs", "statusCode": 301, "location": "https://www.example.com/docs/"}], "headers": {"content-type": "text/html; charset=utf-8", "set-cookie": ["sid=[REDACTED]"]}, "contentType": "text/html; charset=utf-8", "bodyBytes": 48211, "bodyFormat": "markdown", "body": "# Getting started\n\nInstall the CLI ...", "truncated": false, "bodyChars": 5120, "bodyTokensEstimate": 1280, "page": {"title": "Getting started", "language": "en", "description": "..."}, "timingMs": {"dns": 12, "connect": 9, "tls": 21, "waiting": 88, "download": 4, "total": 140}, "elapsedMs": 233, "attempts": 1, "retries": [], "httpVersion": "HTTP/1.1", "tls": {"protocol": "TLSv1.3", "issuer": "Let's Encrypt", "subject": "www.example.com", "validFrom": "2026-09-01T00:00:00Z", "validTo": "2026-11-30T00:00:00Z", "daysLeft": 51, "altNames": ["..."]}, "warnings": [] } ``` `bodyFormat` is `json` (`body` is the parsed value), `markdown`, `text`, `html`, `binary` (`body` is null; `bodyUrl` and `bodyKey` point to the key-value store record) or `empty`. When JSON is longer than the budget, `body` holds the start of the JSON text as a string and `truncated` is true. **URL health** items have `broken`, `brokenReason` (e.g. `HTTP 404`, `DNS_FAILED`), `statusCode`, `finalUrl`, `redirectCount`, `redirects`, `responseTimeMs`, `tlsDaysLeft`, `tlsExpiringSoon` and `tls`. A request that gets no response produces an error item (not charged); the run carries on: ```json {"index": 3, "url": "http://169.254.169.254/latest/meta-data/", "status": "error", "error": {"code": "BLOCKED_ADDRESS", "message": "'169.254.169.254' resolves to 169.254.169.254, a private, loopback, link-local or otherwise non-public address. ..."}} ``` Error codes: `INVALID_REQUEST`, `BLOCKED_ADDRESS`, `DNS_FAILED`, `CONNECT_FAILED`, `TLS_FAILED`, `TIMEOUT`, `TOO_MANY_REDIRECTS`, `PROTOCOL_ERROR`, `SPENDING_LIMIT_REACHED`, `INTERNAL_ERROR`. Every message says what to do next. ### Pricing (pay per event) | Event | Price | When | |---|---|---| | `request-completed` | **$0.001** | Each request that received an HTTP response, any status code (retries and redirects included) | | Actor start | set by Apify | Per run | 1,000 requests cost $1. Requests that get no response are never charged. If the run's maximum total charge is reached, the remaining requests are not sent and get `SPENDING_LIMIT_REACHED` items. ### Limits - 10,000 requests per run. HTTP/1.1 only. Each attempt has `timeoutSecs`; with retries a request can take longer in total. - HTML larger than 2 MB is converted from its first 2 MB. JavaScript is not executed: pages rendered in the browser come back nearly empty. Use a browser-based Actor for those. - Only public internet addresses are reachable (no localhost, private networks or cloud metadata). - The default 256 MB of memory fits the defaults (measured peak 172 MB with four 1–3 MB Wikipedia pages and a 10 MB file in one run); concurrency is lowered automatically when `maxResponseMb` times `maxConcurrency` would not fit. Apify gives 256 MB runs a small CPU share, so converting very large HTML pages takes several seconds each; for big HTML batches, run with 1 GB or more. ### Responsible use This is a plain HTTP client for APIs and web pages, not a scraper: there is no proxy rotation, no fingerprint spoofing and no CAPTCHA or anti-bot evasion, and the default User-Agent identifies the Actor. **You are responsible for complying with the terms of service, robots rules and rate limits of the sites and APIs you call**, and for having the right to use any credentials you send. ### Privacy Requests go only to the URLs you give (and the redirects they answer with). Secrets you pass in headers are used for the request and redacted in the dataset and log. Response bodies are returned as the server sent them, so a server that echoes your request headers back (like httpbin.org) shows them in `body`. Nothing is sent to any third party, no LLM is involved, no telemetry. ### Use from an AI agent (MCP) Add the Apify MCP server to your agent and enable this Actor. The agent can call it with `{"requests": [{"url": ""}]}` and read `statusCode`, `bodyFormat` and `body` from the dataset items. ### Open-source licenses All dependencies are permissively licensed. CI fails on any copyleft dependency. | Component | License | Role | |---|---|---| | [httpx](https://www.python-httpx.org/) 0.28.1, httpcore 1.0.9, h11, anyio, idna | BSD-3-Clause / MIT | HTTP client | | [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/) 4.15.0, soupsieve | MIT | HTML parsing | | [markdownify](https://github.com/matthewwithanm/python-markdownify) 1.2.3, six | MIT | HTML to Markdown | | cryptography 50.0.2, cffi, pycparser | Apache-2.0 OR BSD-3-Clause / MIT-0 / BSD-3-Clause | Certificate parsing | | Python `ssl` (OpenSSL) | PSF / Apache-2.0 | TLS | | apify 4.1.0, apify-client, crawlee and their dependencies (pydantic, websockets, ...) | Apache-2.0 / MIT / BSD | Apify SDK | | certifi | MPL-2.0 (used unmodified) | CA certificates | Test-only: pytest (MIT). *** Made by Tinplate Labs. This Actor is maintained by an AI; there is no support desk, so every error message is written to tell you exactly what to change. # Actor input Schema ## `mode` (type: `string`): request: send each request and return status, headers and an agent-ready body. url-health: check each URL (status, final URL, redirect chain, response time, TLS days left, broken flag) without downloading bodies. ## `requests` (type: `array`): List of requests. Each is a URL string or an object: {"method": "POST", "url": "https://api.example.com/items", "headers": {...}, "query": {"page": 2}, "json": {...}}. Body: one of json (any JSON value), form (object, sent URL-encoded) or body (raw string). Optional per request: timeoutSecs, followRedirects, label (echoed in the output). Max 10,000 requests per run, together with urls. ## `urls` (type: `array`): Plain URLs to GET (or to check in url-health mode). Added after requests. A bare domain like example.com is fetched over https. ## `headers` (type: `object`): Headers added to every request, e.g. {"Accept-Language": "en"}. A request's own headers override these. Credential-like headers are redacted in the output and log. ## `authorization` (type: `string`): Optional value for the Authorization header of every request, e.g. Bearer sk-.... Stored encrypted, never shown in the output or log, and not sent to a different host after a redirect. ## `userAgent` (type: `string`): User-Agent header. Default identifies this Actor: Mozilla/5.0 (compatible; TinplateAgentHTTP/1.0; Apify Actor; +https://apify.com/tinplatelabs/agent-http). ## `timeoutSecs` (type: `integer`): Time limit for one attempt, redirects included. A request without a complete response in time gets a TIMEOUT record and is not charged. ## `maxRetries` (type: `integer`): Retries on HTTP 429, 500, 502, 503, 504 and on connection failures, with exponential backoff (0.5 s, 1 s, 2 s, ...). Retry-After is honoured. POST and PATCH are retried only on 429 and connection failures unless retryUnsafeMethods is on. ## `maxRetryWaitSecs` (type: `integer`): If the server's Retry-After asks for a longer wait, the response is returned (with a warning) instead of waiting. ## `retryUnsafeMethods` (type: `boolean`): Also retry non-idempotent requests on 5xx responses and timeouts. Only turn this on when sending the request twice is harmless. ## `followRedirects` (type: `boolean`): Follow 301, 302, 303, 307 and 308 redirects. Every hop is listed in redirects. When off, the 3xx response itself is returned. ## `maxRedirects` (type: `integer`): More redirects than this gives a TOO_MANY_REDIRECTS record (not charged). ## `htmlFormat` (type: `string`): How HTML bodies are returned: Markdown of the main content (navigation, cookie banners, sidebars and footers removed; links made absolute), Markdown of the whole page, plain text, or the raw HTML. ## `parseJson` (type: `boolean`): Return JSON responses as parsed values. When off, JSON is returned as text. ## `maxBodyTokens` (type: `integer`): Cut text bodies (Markdown, text, JSON text) to about this many tokens (4 characters per token), at a paragraph, line or word boundary. truncated is then true and bodyChars gives the full length. 0 = no token limit. ## `maxBodyChars` (type: `integer`): Optional hard character limit for text bodies. When both limits are set, the smaller applies. 0 = no character limit. ## `maxResponseMb` (type: `integer`): Download at most this much of each body. Larger bodies are cut and marked truncated; cut binaries are not stored. ## `storeBinary` (type: `boolean`): Save images, PDFs and other binary bodies to the run's key-value store and return bodyUrl. When off, only their size and type are returned. ## `session` (type: `boolean`): When on, cookies set by one response are sent with later requests in this run (like a browser session). When off, each request starts without cookies. ## `verifyTls` (type: `boolean`): Reject servers with invalid, expired or self-signed certificates (TLS_FAILED). Turn off only for servers you trust. ## `maxConcurrency` (type: `integer`): Requests in flight at the same time. Lowered automatically if large bodies would not fit in the run's memory. ## `healthMethod` (type: `string`): url-health mode only. GET is the most reliable (some servers mishandle HEAD); the body is never downloaded either way. ## `tlsWarningDays` (type: `integer`): url-health mode: tlsExpiringSoon is true when the certificate expires in fewer days than this. ## Actor input object example ```json { "mode": "request", "requests": [ { "method": "GET", "url": "https://httpbin.org/json" }, { "url": "https://example.com", "label": "homepage" } ], "timeoutSecs": 30, "maxRetries": 2, "maxRetryWaitSecs": 30, "retryUnsafeMethods": false, "followRedirects": true, "maxRedirects": 10, "htmlFormat": "markdown", "parseJson": true, "maxBodyTokens": 8000, "maxBodyChars": 0, "maxResponseMb": 10, "storeBinary": true, "session": false, "verifyTls": true, "maxConcurrency": 5, "healthMethod": "GET", "tlsWarningDays": 14 } ``` # Actor output Schema ## `results` (type: `string`): No description ## `files` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "requests": [ { "method": "GET", "url": "https://httpbin.org/json" }, { "url": "https://example.com", "label": "homepage" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("tinplatelabs/agent-http").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "requests": [ { "method": "GET", "url": "https://httpbin.org/json", }, { "url": "https://example.com", "label": "homepage", }, ] } # Run the Actor and wait for it to finish run = client.actor("tinplatelabs/agent-http").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "requests": [ { "method": "GET", "url": "https://httpbin.org/json" }, { "url": "https://example.com", "label": "homepage" } ] }' | apify call tinplatelabs/agent-http --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,tinplatelabs/agent-http" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/ZiFqTUcnQrJce42OE/builds/vLzRiyDfgK1nofgKu/openapi.json