HTTP Request Tool for AI Agents

Give it a batch of HTTP requests and get back one clean, agent-ready record per request: status, headers, the body as parsed JSON, Markdown of the page's main content (for HTML) or text, the full redirect chain, a timing breakdown and TLS certificate details. Built-in retries, timeouts, size limits and SSRF protection. A URL health mode checks lists of links.

Built for AI agents (Claude, ChatGPT, Cursor and others through the Apify MCP server) that need to call APIs or read web pages without blowing up their context window, and for anyone who needs a reliable HTTP client in an Apify workflow.

Why this one

• Bodies an agent can read. JSON comes back parsed. HTML becomes Markdown of the main content, with navigation, cookie banners, sidebars and footers stripped and links made absolute. Binary files (images, PDFs, ZIPs) are saved to the key-value store and returned as a URL instead of a blob.
• Token budget. maxBodyTokens (default 8,000) cuts long bodies at a paragraph boundary and says so (truncated: true, bodyChars, bodyTokensEstimate), so one huge page can't flood the context.
• Robust. Retries on 429/5xx and connection failures with exponential backoff, honouring Retry-After. POST/PATCH are only retried when it's safe (429 or connection failure) unless you allow it.
• Transparent. Every redirect hop, attempt and retry is listed; timing is split into DNS, connect, TLS, waiting (time to first byte) and download.
• Safe by default. Private, loopback, link-local and cloud-metadata addresses (e.g. 169.254.169.254) are refused, including after redirects and DNS resolution. Authorization, cookies and API-key headers and query parameters are redacted in the output and log, and credentials are not forwarded to another host on a redirect. No cookies are kept between requests unless you turn on session.
• Pay only for responses. Any HTTP response is charged, including 404 or 500 (that's a successful request). DNS failures, timeouts, blocked addresses and invalid input cost nothing.

Input

{
  "requests": [
    {"method": "GET", "url": "https://httpbin.org/json"},
    {"method": "POST", "url": "https://api.example.com/v1/search", "json": {"q": "widgets"},
     "headers": {"X-Api-Key": "..."}, "label": "search"},
    "https://example.com"
  ],
  "authorization": "Bearer sk-...",
  "maxBodyTokens": 4000
}

Field	Default	What it does
mode	request	request returns full responses; url-health checks links without downloading bodies
requests		URL strings or objects: method, url, headers, query, one of json / form / body, timeoutSecs, followRedirects, label
urls		Plain URLs to GET (or check). Bare domains get https://
headers		Headers for every request (a request's own headers win)
authorization		Secret Authorization value for every request (stored encrypted, never output)
userAgent	identifies this Actor	User-Agent header
timeoutSecs	30	Time limit per attempt, redirects included (1–300)
maxRetries	2	Retries on 429/500/502/503/504 and connection failures (0–5)
maxRetryWaitSecs	30	Longest Retry-After to wait for
retryUnsafeMethods	false	Also retry POST/PATCH on 5xx and timeouts
followRedirects / maxRedirects	true / 10	Redirect handling
htmlFormat	markdown	markdown (main content), markdown-full, text or html
parseJson	true	Return JSON parsed
maxBodyTokens / maxBodyChars	8000 / 0	Body budget (≈4 characters per token); 0 = no limit; the smaller applies
maxResponseMb	10	Download limit per body (1–50)
storeBinary	true	Save binary bodies to the key-value store
session	false	Keep cookies across the run's requests
verifyTls	true	Reject invalid certificates
maxConcurrency	5	Requests in flight (1–20)
healthMethod / tlsWarningDays	GET / 14	url-health mode options

Output

One dataset item per request, in completion order (index is the position in the input). Abridged:

{
  "index": 0, "label": "docs", "method": "GET", "url": "https://example.com/docs",
  "status": "ok", "statusCode": 200, "statusText": "OK", "ok": true,
  "finalUrl": "https://www.example.com/docs/",
  "redirects": [{"url": "https://example.com/docs", "statusCode": 301, "location": "https://www.example.com/docs/"}],
  "headers": {"content-type": "text/html; charset=utf-8", "set-cookie": ["sid=[REDACTED]"]},
  "contentType": "text/html; charset=utf-8",
  "bodyBytes": 48211,
  "bodyFormat": "markdown",
  "body": "# Getting started\n\nInstall the CLI ...",
  "truncated": false, "bodyChars": 5120, "bodyTokensEstimate": 1280,
  "page": {"title": "Getting started", "language": "en", "description": "..."},
  "timingMs": {"dns": 12, "connect": 9, "tls": 21, "waiting": 88, "download": 4, "total": 140},
  "elapsedMs": 233, "attempts": 1, "retries": [], "httpVersion": "HTTP/1.1",
  "tls": {"protocol": "TLSv1.3", "issuer": "Let's Encrypt", "subject": "www.example.com",
          "validFrom": "2026-09-01T00:00:00Z", "validTo": "2026-11-30T00:00:00Z", "daysLeft": 51, "altNames": ["..."]},
  "warnings": []
}

bodyFormat is json (body is the parsed value), markdown, text, html, binary (body is null; bodyUrl and bodyKey point to the key-value store record) or empty. When JSON is longer than the budget, body holds the start of the JSON text as a string and truncated is true.

URL health items have broken, brokenReason (e.g. HTTP 404, DNS_FAILED), statusCode, finalUrl, redirectCount, redirects, responseTimeMs, tlsDaysLeft, tlsExpiringSoon and tls.

A request that gets no response produces an error item (not charged); the run carries on:

{"index": 3, "url": "http://169.254.169.254/latest/meta-data/", "status": "error",
 "error": {"code": "BLOCKED_ADDRESS", "message": "'169.254.169.254' resolves to 169.254.169.254, a private, loopback, link-local or otherwise non-public address. ..."}}

Error codes: INVALID_REQUEST, BLOCKED_ADDRESS, DNS_FAILED, CONNECT_FAILED, TLS_FAILED, TIMEOUT, TOO_MANY_REDIRECTS, PROTOCOL_ERROR, SPENDING_LIMIT_REACHED, INTERNAL_ERROR. Every message says what to do next.

Pricing (pay per event)

Event	Price	When
request-completed	$0.001	Each request that received an HTTP response, any status code (retries and redirects included)
Actor start	set by Apify	Per run

1,000 requests cost $1. Requests that get no response are never charged. If the run's maximum total charge is reached, the remaining requests are not sent and get SPENDING_LIMIT_REACHED items.

Limits

• 10,000 requests per run. HTTP/1.1 only. Each attempt has timeoutSecs; with retries a request can take longer in total.
• HTML larger than 2 MB is converted from its first 2 MB. JavaScript is not executed: pages rendered in the browser come back nearly empty. Use a browser-based Actor for those.
• Only public internet addresses are reachable (no localhost, private networks or cloud metadata).
• The default 256 MB of memory fits the defaults (measured peak 172 MB with four 1–3 MB Wikipedia pages and a 10 MB file in one run); concurrency is lowered automatically when maxResponseMb times maxConcurrency would not fit. Apify gives 256 MB runs a small CPU share, so converting very large HTML pages takes several seconds each; for big HTML batches, run with 1 GB or more.

Responsible use

This is a plain HTTP client for APIs and web pages, not a scraper: there is no proxy rotation, no fingerprint spoofing and no CAPTCHA or anti-bot evasion, and the default User-Agent identifies the Actor. You are responsible for complying with the terms of service, robots rules and rate limits of the sites and APIs you call, and for having the right to use any credentials you send.

Privacy

Requests go only to the URLs you give (and the redirects they answer with). Secrets you pass in headers are used for the request and redacted in the dataset and log. Response bodies are returned as the server sent them, so a server that echoes your request headers back (like httpbin.org) shows them in body. Nothing is sent to any third party, no LLM is involved, no telemetry.

Use from an AI agent (MCP)

Add the Apify MCP server to your agent and enable this Actor. The agent can call it with {"requests": [{"url": "<url>"}]} and read statusCode, bodyFormat and body from the dataset items.

Open-source licenses

All dependencies are permissively licensed. CI fails on any copyleft dependency.

Component	License	Role
httpx 0.28.1, httpcore 1.0.9, h11, anyio, idna	BSD-3-Clause / MIT	HTTP client
Beautiful Soup 4.15.0, soupsieve	MIT	HTML parsing
markdownify 1.2.3, six	MIT	HTML to Markdown
cryptography 50.0.2, cffi, pycparser	Apache-2.0 OR BSD-3-Clause / MIT-0 / BSD-3-Clause	Certificate parsing
Python ssl (OpenSSL)	PSF / Apache-2.0	TLS
apify 4.1.0, apify-client, crawlee and their dependencies (pydantic, websockets, ...)	Apache-2.0 / MIT / BSD	Apify SDK
certifi	MPL-2.0 (used unmodified)	CA certificates

Test-only: pytest (MIT).

---

Made by Tinplate Labs. This Actor is maintained by an AI; there is no support desk, so every error message is written to tell you exactly what to change.