Web Fetch API — Fetch Any URL as HTML, Text, Markdown or JSON

Bulk HTTP fetch built for AI agents and pipelines: give it a list of URLs and get back, for each one, the HTTP status, the response headers and the body — raw HTML, plain text, Markdown or parsed JSON. Follows redirects, supports GET/POST/HEAD and custom headers, and can render JavaScript pages in a headless browser when you ask it to.

Try it in 1 minute:

1. Press Start — the prefilled input fetches example.com, a public JSON API and a Wikipedia article.
2. Wait a few seconds.
3. Open the Output tab: one record per URL with httpStatus, headers, contentType, body (the JSON answer comes back parsed, the HTML pages raw) plus title, language and word count for HTML pages.

Price: $0.30 per 1,000 pages. URLs that cannot be fetched (404, timeouts, refused requests, blocked by robots.txt) are free.

Use it to:

• Give an AI agent a reliable "fetch this URL" tool through the Apify MCP server or the API — one call, a clean JSON record back.
• Pull hundreds of pages or API endpoints in one batch instead of writing retry and timeout logic yourself.
• Convert HTML pages to Markdown or plain text on the fly (format: markdown) for LLM prompts and RAG.
• Check status codes, redirects and response headers of a URL list (monitoring, link audits, SEO checks).
• Read JSON APIs and feeds into a dataset you can export as CSV or Excel.

How it works

Every URL is requested with browser-like default headers (override any of them), a per-request timeout and automatic redirect following. The answer is kept whole: status code, headers, final URL, size and duration. The body is delivered in the format you choose — auto parses JSON content types, keeps HTML raw and returns everything else as text; markdown and text run an HTML page through Mozilla Readability (main content only, unless you turn that off) and Turndown. A 5xx or 429 answer is retried once through the Apify proxy; a network error too. Binary content (images, PDFs, archives) is reported with its size and content type but not delivered as a body.

Use a browser renders GET requests in headless Chromium first, for pages that build their content with JavaScript. It is slower and costs more compute per page, so leave it off unless a plain fetch comes back without the content you need.

robots.txt is respected by default: a URL the site disallows for bots is skipped with a free ROBOTS_DISALLOWED record. Turn the check off for URLs you have the right to fetch, such as your own API.

Input

Field	Default	What it does
urls	—	Addresses to fetch, one per line (required)
method	GET	GET, POST, HEAD, PUT, PATCH, DELETE or OPTIONS
headers	{}	Extra request headers (JSON object); they replace defaults of the same name
format	auto	auto, html, text, markdown or json
useBrowser	false	Render GET requests in headless Chromium
body	—	Request body for POST/PUT/PATCH
followRedirects	true	Follow 3xx and report the final URL
timeoutSecs	30	Per-request timeout
respectRobotsTxt	true	Skip URLs the site's robots.txt disallows
mainContentOnly	true	text/markdown: Readability main content only
includeHeaders	true	Deliver response headers
maxBodyLength	0	Cut text bodies after N characters
maxConcurrency, proxyConfiguration	5, Apify proxy	Speed and retry proxy

Output

Field	What it is
url, finalUrl, redirected, method	what was requested and where it ended
httpStatus, headers, contentType, bytes, durationMs	the answer
format, body, formatError	the body as html / text / markdown / parsed json / binary (null)
title, description, language, canonicalUrl, wordCount, linksCount	HTML pages only
renderedWithBrowser, truncated	how the record was made
input, status, error, scrapedAt	what you gave, OK or the reason, and when

Status codes: OK (any 2xx or final 3xx answer), NOT_FOUND (404/410), ROBOTS_DISALLOWED, INVALID_INPUT, FAILED (network error, timeout, other 4xx, 5xx after retries). Only OK records are charged.

Example

{
  "input": "https://jsonplaceholder.typicode.com/posts/1",
  "url": "https://jsonplaceholder.typicode.com/posts/1",
  "finalUrl": "https://jsonplaceholder.typicode.com/posts/1",
  "redirected": false,
  "method": "GET",
  "httpStatus": 200,
  "contentType": "application/json",
  "headers": { "content-type": "application/json; charset=utf-8", "cache-control": "max-age=43200" },
  "format": "json",
  "body": { "userId": 1, "id": 1, "title": "sunt aut facere repellat provident", "body": "quia et suscipit..." },
  "bytes": 292,
  "durationMs": 184,
  "renderedWithBrowser": false,
  "status": "OK",
  "scrapedAt": "2026-10-19T12:00:00.000Z"
}

FAQ

Is this an official API? No, it's an unofficial API built on public pages.

Does it log in, solve captchas or get around bot walls? No. It fetches public URLs only. A 401 or 403 answer is reported as FAILED with the status, free of charge.

How is markdown different from html? html is the page as the server sent it. markdown keeps only the main content (Readability) and converts it to Markdown with headings, lists, links and tables — a fraction of the tokens for an LLM.

Why did a page come back without the content I see in my browser? The site renders it with JavaScript. Turn on Use a browser.

Can I send POST requests with a JSON body? Yes: set method to POST, put the JSON text in body and add {"Content-Type": "application/json"} to headers.

Like it? A short review on the Store page helps other people find this Actor. Something missing or broken? Tell us on the Issues tab — we read every one.

Related Actors

• Website Tech Stack Detector — CMS, frameworks and analytics of any site
• Sitemap Extractor & Scraper — every URL, date and image from a sitemap
• Domain Checker — WHOIS, DNS, SSL and expiry dates in bulk
• URL to Markdown API — clean Markdown from any URL for LLMs
• Markdown Crawler API — crawl a whole website to Markdown for RAG