HTTP Request & Bulk API Calls: Extract JSON, URL to Markdown

Send HTTP requests (GET, POST, PUT, PATCH, DELETE) to any API or web page, one or 10,000 per run, and get one clean result per request: status, headers, parsed JSON, the JSON fields you pick as flat columns, or a web page as clean Markdown. Bulk API calls from a CSV, Google Sheet or Apify dataset: write the request once with {{column}} placeholders and it is sent for every row, with your row's ID kept on each result. Retries, redirects, timing and TLS details included. $1 per 1,000 responses; failed requests are free.

What people use it for

Use case	How
Call an API from Clay, n8n, Make or Zapier without a paid HTTP plan	requests + extract to get just the columns you need
Give an AI agent (Claude, ChatGPT, Cursor via the Apify MCP server) a safe HTTP tool	JSON parsed, HTML as Markdown, a token budget so one page can't flood the context
Bulk API calls from CSV: call an API for each row of a CSV, Google Sheet or dataset (API enrichment, posting records)	requestTemplate with {{column}} placeholders + rowsUrl / rowsDatasetId; passThroughColumns joins results back
Bulk API calls from a list: post records, poll job status	Up to 10,000 requests per run, 5–20 in parallel, retries with Retry-After
URL to Markdown for RAG	HTML becomes Markdown of the main content, navigation and cookie banners stripped
Broken link checker	mode: url-health: status, final URL, redirect chain, response time, TLS days left

Quick start (real output)

This input (the form's prefill) finishes in about 2 seconds and costs $0.003:

{
  "requests": [
    {"url": "https://jsonplaceholder.typicode.com/users/1", "label": "user",
     "extract": {"name": "name", "email": "email", "city": "address.city", "company": "company.name"}},
    {"method": "POST", "url": "https://jsonplaceholder.typicode.com/posts", "label": "create post",
     "json": {"title": "hello", "body": "sent by an agent", "userId": 1}, "extract": {"newId": "id"}},
    {"url": "https://example.com", "label": "web page as Markdown"}
  ]
}

Output, abridged to the interesting fields:

[
  {"label": "user", "statusCode": 200, "bodyFormat": "json",
   "extracted": {"name": "Leanne Graham", "email": "Sincere@april.biz", "city": "Gwenborough", "company": "Romaguera-Crona"},
   "extractMissing": []},
  {"label": "create post", "method": "POST", "statusCode": 201, "statusText": "Created",
   "body": {"title": "hello", "body": "sent by an agent", "userId": 1, "id": 101}, "extracted": {"newId": 101}},
  {"label": "web page as Markdown", "statusCode": 200, "bodyFormat": "markdown",
   "body": "This domain is for use in documentation examples without needing permission. ...",
   "page": {"title": "Example Domain", "language": "en"}}
]

Set "includeBody": false to drop the body and keep only status, headers and extracted: one small row per call.

Bulk API calls from CSV, Google Sheet or dataset rows

Call an API once for every row: write one requestTemplate with {{column}} placeholders and give the rows as an Apify dataset (rowsDatasetId, e.g. a scraper's output), a CSV / JSON / Google Sheets link (rowsUrl) or inline rows. Placeholders work in the URL, query, headers and body.

{
  "requestTemplate": {
    "method": "POST",
    "url": "https://httpbin.org/anything/companies/{{domain}}",
    "query": {"country": "{{country}}"},
    "json": {"name": "{{name}}", "employees": "{{employees}}"},
    "extract": {"sentName": "json.name", "sentEmployees": "json.employees", "path": "url"}
  },
  "rows": [
    {"id": "c-1", "domain": "acme.com", "name": "Acme \"Rockets\" Inc.", "country": "US", "employees": 120},
    {"id": "c-2", "domain": "globex.io", "name": "Globex", "country": "DE", "employees": 40}
  ],
  "passThroughColumns": ["id"],
  "includeBody": false
}

Each result keeps rowIndex and the columns you pass through, so it joins straight back to your sheet or CRM:

{"index": 0, "rowIndex": 0, "row": {"id": "c-1"}, "method": "POST",
 "url": "https://httpbin.org/anything/companies/acme.com?country=US", "statusCode": 200,
 "extracted": {"sentName": "Acme \"Rockets\" Inc.", "sentEmployees": 120, "path": "https://httpbin.org/anything/companies/acme.com?country=US"}}

• Escaping is automatic. Values are URL-encoded in the URL (a b/c becomes a%20b%2Fc) and JSON-escaped in bodies, so quotes, slashes and line breaks in your data can't break a request. In json, a value that is just "{{column}}" keeps the column's type (number, true/false, list). A placeholder at the very start of the URL ({{url}}, for a column of full URLs) or in triple braces ({{{path}}}) is inserted as is.
• Columns: by name, or a dot path for nested dataset items ({{address.city}}). Plain lists (

  ["a.com", "b.com"]
  ) are available as {{value}}. CSV and TSV files need a header row; JSON arrays, JSON Lines and {"items": [...]} files work too. A Google Sheet must be shared as "Anyone with the link can view".
• Missing data is flagged, not guessed: a row without a column the template uses gets a free MISSING_COLUMN item that names the column and lists the row's columns.
• Duplicates are sent once (dedupeRequests, on by default): repeats get a free skipped item with duplicateOfIndex.
• Test on a few rows first with maxRows; for more than 10,000 rows, run again with rowsOffset.

Pricing in plain words

$0.001 per request that gets an HTTP response (any status, 404 and 500 included; retries and redirects included). DNS failures, timeouts, blocked addresses and invalid input are never charged. 1,000 API calls = $1. JSON extraction, Markdown conversion, link checks, reading your CSV or dataset rows, skipped duplicates and rows with missing columns cost nothing extra.

Why this one

• Extract JSON fields. extract maps output names to paths (data.items[0].id, items[-1].name, items[*].id, ["key.with.dots"]). It reads the whole JSON, even when the body is larger than the token budget, so you get your fields from big API responses too.
• One API call per CSV or dataset row. {{column}} templates with automatic URL and JSON escaping, pass-through columns to join results back, duplicate skipping and clear errors for missing columns.
• Bodies an agent can read. JSON comes back parsed. HTML becomes Markdown of the main content, with navigation, cookie banners, sidebars and footers stripped and links made absolute. Binary files (images, PDFs, ZIPs) are saved to the key-value store and returned as a URL instead of a blob.
• Token budget. maxBodyTokens (default 8,000) cuts long bodies at a paragraph boundary and says so (truncated: true, bodyChars, bodyTokensEstimate), so one huge page can't flood the context.
• Robust. Retries on 429/5xx and connection failures with exponential backoff, honouring Retry-After. POST/PATCH are only retried when it's safe (429 or connection failure) unless you allow it.
• Transparent. Every redirect hop, attempt and retry is listed; timing is split into DNS, connect, TLS, waiting (time to first byte) and download.
• Safe by default. Private, loopback, link-local and cloud-metadata addresses (e.g. 169.254.169.254) are refused, including after redirects and DNS resolution. Authorization, cookies and API-key headers and query parameters are redacted in the output and log, and credentials are not forwarded to another host on a redirect. No cookies are kept between requests unless you turn on session.
• Pay only for responses. A request that gets no response costs nothing.

Input

{
  "requests": [
    {"method": "GET", "url": "https://httpbin.org/json"},
    {"method": "POST", "url": "https://api.example.com/v1/search", "json": {"q": "widgets"},
     "headers": {"X-Api-Key": "..."}, "label": "search"},
    "https://example.com"
  ],
  "authorization": "Bearer sk-...",
  "maxBodyTokens": 4000
}

Field	Default	What it does
mode	request	request returns full responses; url-health checks links without downloading bodies
requests		URL strings or objects: method, url, headers, query, one of json / form / body, extract, timeoutSecs, followRedirects, label
urls		Plain URLs to GET (or check). Bare domains get https://
extract		JSON fields to pick for every request: {"name": "path"}; a request's own extract overrides names
includeBody	true	false drops the body (bodyFormat: "omitted") so items hold status, headers and extracted only
headers		Headers for every request (a request's own headers win)
authorization		Secret Authorization value for every request (stored encrypted, never output)
userAgent	identifies this Actor	User-Agent header
timeoutSecs	30	Time limit per attempt, redirects included (1–300)
maxRetries	2	Retries on 429/500/502/503/504 and connection failures (0–5)
maxRetryWaitSecs	30	Longest Retry-After to wait for
retryUnsafeMethods	false	Also retry POST/PATCH on 5xx and timeouts
followRedirects / maxRedirects	true / 10	Redirect handling
htmlFormat	markdown	markdown (main content), markdown-full, text or html
parseJson	true	Return JSON parsed
maxBodyTokens / maxBodyChars	8000 / 0	Body budget (≈4 characters per token); 0 = no limit; the smaller applies
maxResponseMb	10	Download limit per body (1–50)
storeBinary	true	Save binary bodies to the key-value store
session	false	Keep cookies across the run's requests
verifyTls	true	Reject invalid certificates
maxConcurrency	5	Requests in flight (1–20)
healthMethod / tlsWarningDays	GET / 14	url-health mode options
requestTemplate		One request object with {{column}} placeholders, sent for every row (after requests and urls)
rowsDatasetId / rowsUrl / rows		The rows: an Apify dataset, a CSV/TSV/JSON/JSON Lines/Google Sheets URL (≤25 MB), or inline. Give one
passThroughColumns		Row columns copied into each result as row (* = all)
maxRows / rowsOffset	0 / 0	Send at most this many rows (0 = all) / skip this many first
dedupeRequests	true	Send identical rendered requests once

Output

One dataset item per request, in completion order (index is the position in the input). Abridged:

{
  "index": 0, "label": "docs", "method": "GET", "url": "https://example.com/docs",
  "status": "ok", "statusCode": 200, "statusText": "OK", "ok": true,
  "finalUrl": "https://www.example.com/docs/",
  "redirects": [{"url": "https://example.com/docs", "statusCode": 301, "location": "https://www.example.com/docs/"}],
  "headers": {"content-type": "text/html; charset=utf-8", "set-cookie": ["sid=[REDACTED]"]},
  "contentType": "text/html; charset=utf-8",
  "bodyBytes": 48211,
  "bodyFormat": "markdown",
  "body": "# Getting started\n\nInstall the CLI ...",
  "truncated": false, "bodyChars": 5120, "bodyTokensEstimate": 1280,
  "page": {"title": "Getting started", "language": "en", "description": "..."},
  "timingMs": {"dns": 12, "connect": 9, "tls": 21, "waiting": 88, "download": 4, "total": 140},
  "elapsedMs": 233, "attempts": 1, "retries": [], "httpVersion": "HTTP/1.1",
  "tls": {"protocol": "TLSv1.3", "issuer": "Let's Encrypt", "subject": "www.example.com",
          "validFrom": "2026-09-01T00:00:00Z", "validTo": "2026-11-30T00:00:00Z", "daysLeft": 51, "altNames": ["..."]},
  "warnings": []
}

bodyFormat is json (body is the parsed value), markdown, text, html, binary (body is null; bodyUrl and bodyKey point to the key-value store record), empty or omitted (includeBody: false). With extract, items also have extracted (name to value; null when the path matches nothing) and extractMissing (those names). Path syntax: dot keys, [n] or .n index (negative counts from the end), [*] or * for every element (gives a list), ["key.with.dots"], optional $. prefix. JSON Lines bodies are read as a list. A non-JSON response gives extracted: null and a warning (still charged: it got a response). When JSON is longer than the budget, body holds the start of the JSON text as a string and truncated is true.

URL health items have broken, brokenReason (e.g. HTTP 404, DNS_FAILED), statusCode, finalUrl, redirectCount, redirects, responseTimeMs, tlsDaysLeft, tlsExpiringSoon and tls.

Template rows add rowIndex (the row's position in its source) and, with passThroughColumns, row. Repeated requests give {"status": "skipped", "skipReason": "DUPLICATE_REQUEST", "duplicateOfIndex": 0}.

A request that gets no response produces an error item (not charged); the run carries on:

{"index": 3, "url": "http://169.254.169.254/latest/meta-data/", "status": "error",
 "error": {"code": "BLOCKED_ADDRESS", "message": "'169.254.169.254' resolves to 169.254.169.254, a private, loopback, link-local or otherwise non-public address. ..."}}

Error codes: INVALID_REQUEST, BLOCKED_ADDRESS, DNS_FAILED, CONNECT_FAILED, TLS_FAILED, TIMEOUT, TOO_MANY_REDIRECTS, PROTOCOL_ERROR, SPENDING_LIMIT_REACHED, INTERNAL_ERROR, and for templates MISSING_COLUMN (a row lacks a placeholder's column) and ROWS_UNREADABLE (the dataset or file could not be read). Every message says what to do next.

Pricing (pay per event)

Event	Price	When
request-completed	$0.001	Each request that received an HTTP response, any status code (retries and redirects included)
Actor start	set by Apify	Per run

1,000 requests cost $1. Requests that get no response are never charged. If the run's maximum total charge is reached, the remaining requests are not sent and get SPENDING_LIMIT_REACHED items.

Limits

• 10,000 requests per run (template rows included; continue with rowsOffset). Row files up to 25 MB. HTTP/1.1 only. Each attempt has timeoutSecs; with retries a request can take longer in total.
• HTML larger than 2 MB is converted from its first 2 MB. JavaScript is not executed: pages rendered in the browser come back nearly empty. Use a browser-based Actor for those.
• Only public internet addresses are reachable (no localhost, private networks or cloud metadata).
• The default 256 MB of memory fits the defaults (measured peak 172 MB with four 1–3 MB Wikipedia pages and a 10 MB file in one run); concurrency is lowered automatically when maxResponseMb times maxConcurrency would not fit. Apify gives 256 MB runs a small CPU share, so converting very large HTML pages takes several seconds each; for big HTML batches, run with 1 GB or more.

Responsible use

This is a plain HTTP client for APIs and web pages, not a scraper: there is no proxy rotation, no fingerprint spoofing and no CAPTCHA or anti-bot evasion, and the default User-Agent identifies the Actor. You are responsible for complying with the terms of service, robots rules and rate limits of the sites and APIs you call, and for having the right to use any credentials you send.

Privacy

Requests go only to the URLs you give (and the redirects they answer with). Secrets you pass in headers are used for the request and redacted in the dataset and log. Response bodies are returned as the server sent them, so a server that echoes your request headers back (like httpbin.org) shows them in body. Nothing is sent to any third party, no LLM is involved, no telemetry.

Use from an AI agent (MCP)

Add the Apify MCP server to your agent and enable this Actor. The agent can call it with {"requests": [{"url": "<url>"}]} and read statusCode, bodyFormat and body from the dataset items. For big API responses, pass extract with the paths it needs and "includeBody": false to keep the context small.

Open-source licenses

All dependencies are permissively licensed. CI fails on any copyleft dependency.

Component	License	Role
httpx 0.28.1, httpcore 1.0.9, h11, anyio, idna	BSD-3-Clause / MIT	HTTP client
Beautiful Soup 4.15.0, soupsieve	MIT	HTML parsing
markdownify 1.2.3, six	MIT	HTML to Markdown
cryptography 50.0.2, cffi, pycparser	Apache-2.0 OR BSD-3-Clause / MIT-0 / BSD-3-Clause	Certificate parsing
Python ssl (OpenSSL)	PSF / Apache-2.0	TLS
apify 4.1.0, apify-client, crawlee and their dependencies (pydantic, websockets, ...)	Apache-2.0 / MIT / BSD	Apify SDK
certifi	MPL-2.0 (used unmodified)	CA certificates

Test-only: pytest (MIT).

---

Made by Tinplate Labs. This Actor is maintained by an AI; there is no support desk, so every error message is written to tell you exactly what to change.