Agent Web Reader: web search + URL to clean markdown for AI agents

Why agents pick this

• $0.003 per search, $0.002 per page read. A query plus 3 pages costs $0.010. 404s, timeouts, blocks and empty pages are returned as rows with an errorCode and are never charged.
• One flat row per page, with url, title, markdown, status. That makes citations trivial and needs no HTML clean-up.
• Fits your context window. maxCharsPerResult cuts at a paragraph boundary, and summary mode returns less.
• One bad link never sinks the batch. Every URL gets its own status row.

What it does: give it a search query and/or a list of urls; get back one JSON row per page with clean main-content markdown, title, source url, status and, on failure, an errorCode + errorMessage. Reads HTML pages and PDFs. Built for LLM tool calls: small outputs, a per-result character budget, and a summary mode.

Use it when an agent needs to: answer a question from the live web with citations, read a link a user pasted, read a PDF from a URL, or triage several pages cheaply.

Don't use it for: pages behind a login or paywall, sites that only render with heavy JavaScript (SPAs; you get EMPTY_CONTENT), crawling a whole site (use a crawler), screenshots (use catalyst-labs/website-screenshot-pdf-api), or DOCX/XLSX files (use catalyst-labs/document-to-markdown).

Cost per call

Event	Price	When
search	$0.003	once per query that returned results
page	$0.002	per page or PDF read successfully
actor-start	$0.001	once per batch run (not charged in Standby/HTTP mode)

Typical call: query + 3 results = $0.001 + $0.003 + 3 x $0.002 = $0.010. Reading one URL = $0.003. Errors (404, timeout, blocked, empty, scanned PDF) are never charged. Set maxTotalChargeUsd on the run to cap spend; when the cap is reached remaining URLs return SPENDING_LIMIT rows instead of being fetched.

Input (only query or urls is required)

{ "query": "what is the model context protocol", "maxResults": 3, "maxCharsPerResult": 4000 }

{ "urls": ["https://arxiv.org/pdf/1706.03762", "https://en.wikipedia.org/wiki/Model_Context_Protocol"], "outputMode": "summary" }

Field	Default	Notes
query	–	Web search; top maxResults results are opened.
urls	–	Up to 50 URLs (HTML or PDF). Combine with query if you like.
maxResults	3	1-10 search results to read.
maxCharsPerResult	6000	Token budget per result (~4 chars per token). Cut on a paragraph boundary.
outputMode	markdown	summary = lead paragraphs + section outline, max 1,500 chars.
includeLinks	false	Keep [text](url) links.
fullPage	false	Whole page body instead of main-article extraction (list/index pages).
country	–	2-letter code for localized search.
maxPdfPages	50	Pages read per PDF.
timeoutSecs	15	Per-URL timeout; slow URLs become error rows.

Output (one flat row per page; same fields every time)

{
  "status": "ok",
  "url": "https://en.wikipedia.org/wiki/Model_Context_Protocol",
  "finalUrl": "https://en.wikipedia.org/wiki/Model_Context_Protocol",
  "title": "Model Context Protocol",
  "markdown": "The Model Context Protocol (MCP) is an open standard ...",
  "chars": 4000, "totalChars": 16749, "truncated": true,
  "source": "search", "searchRank": 1, "query": "what is the model context protocol",
  "contentType": "text/html", "fetchedAt": "2026-10-25T05:01:12.000Z", "ms": 384,
  "errorCode": null, "errorMessage": null
}

{ "status": "error", "url": "https://example.com/missing", "markdown": null, "errorCode": "HTTP_404", "errorMessage": "The server answered 404." }

Error codes: INVALID_URL, DNS_NOT_FOUND, TIMEOUT, HTTP_403/HTTP_429 (site blocked bots), HTTP_404, NO_TEXT_LAYER (scanned PDF), EMPTY_CONTENT, UNSUPPORTED_TYPE, TOO_LARGE, PARSE_FAILED, SEARCH_FAILED, SPENDING_LIMIT. One bad URL never fails the run. The OUTPUT key-value record holds all rows plus searchEngine, okCount, errorCount and chargedEvents.

Fast calls: Standby (HTTP) mode

The Actor runs in Apify Standby mode, so it answers like an API with no cold start per call:

curl "https://catalyst-labs--agent-web-reader.apify.actor/?query=apify%20mcp&maxResults=2&token=$APIFY_TOKEN"
curl -X POST "https://catalyst-labs--agent-web-reader.apify.actor/?token=$APIFY_TOKEN" -H 'content-type: application/json' -d '{"urls":["https://example.com"]}'

Response: { "status": "ok", "query": ..., "searchEngine": ..., "results": [ ...rows... ] }.

Use from MCP (Claude, Cursor, ChatGPT, any MCP client)

https://mcp.apify.com/?tools=catalyst-labs/agent-web-reader (Apify token as Bearer auth). The tool appears as catalyst-labs--agent-web-reader.

How it works

Search: Bing, with DuckDuckGo and Google (Apify SERP proxy) fallbacks; searchEngine tells you which answered. Pages are fetched directly with a datacenter-proxy retry when a site blocks; main content is extracted (Mozilla Readability) and converted to GitHub-flavored markdown (tables kept); scripts, nav, footers and images are dropped (image alt text kept). PDFs are converted page by page with ## Page N headings.

Same input gives the same rows in the same order (search ranks can change over time as the web does). Respect the sites you read; this tool does not bypass logins or paywalls.

Made by Catalyst Labs. Also see: YouTube Transcript API, Document to Markdown, Google Trends API, Website Screenshot & PDF API.