LLM Reader & Web2Markdown

Convert one public web page into compact, main-content Markdown for LLMs, RAG ingestion, AI agents, and downstream content pipelines. This web page-to-Markdown API combines a lightweight HTTP fetch with an SSRF-hardened Playwright fallback for JavaScript-rendered pages.

The Actor uses Mozilla Readability to identify the primary article content, filters common page furniture, and uses Turndown to produce consistent Markdown plus structured source metadata and an approximate token count. It processes exactly one URL per run; it is an article content extractor and HTML-to-Markdown scraper, not a whole-site crawler.

Common use cases

• Prepare a public article or documentation page for chunking, embeddings, and a RAG knowledge base.
• Give an AI agent clean page text, title, author, source URL, and extraction metadata without passing raw HTML.
• Normalize blog posts, help-center articles, or public reports before summarization or classification.
• Extract readable content from a JavaScript-rendered page when the site's network behavior is compatible with the Actor's safety policy.
• Feed a downstream content-monitoring or diff pipeline with repeatable Markdown snapshots, one URL at a time.

The Actor does not crawl links, discover sitemaps, process PDFs, or bypass authentication, CAPTCHAs, paywalls, access controls, or anti-bot protections.

Input

A minimal run only needs a public URL:

{
  "url": "https://example.com/article"
}

All supported options:

{
  "url": "https://example.com/article",
  "render_js": false,
  "browser_fallback": true,
  "min_content_chars": 500,
  "include_links": false,
  "include_images": false,
  "request_timeout_secs": 7,
  "browser_timeout_secs": 20,
  "overall_timeout_secs": 20,
  "max_content_bytes": 3000000
}

Field	Type	Required	Default	Allowed values and behavior
url	string	Yes	—	Absolute public http:// or https:// URL, at most 2,048 characters; URL fragments are discarded. URL credentials, local hostnames, private/link-local/loopback/special IP addresses, mixed public-private DNS answers, nonstandard ports, and unsafe redirects are rejected. Only ports 80 and 443 are supported.
render_js	boolean	No	false	When true, launch Playwright immediately. When false, try lightweight HTTP extraction first.
browser_fallback	boolean	No	true	When render_js is false, allow Playwright if lightweight extraction fails recoverably or is too short. Set to false for a strict fetch-only run.
min_content_chars	integer	No	500	From 100 to 10,000. Lightweight content below this text-character threshold triggers browser fallback when enabled. Content must also contain at least 20 words to be considered sufficient.
include_links	boolean	No	false	Keep filtered HTTP(S) and mailto: references. Relative links are resolved, but retained public-looking hostnames are not DNS-validated; validate them before any downstream fetch.
include_images	boolean	No	false	Keep filtered image references in Markdown. The browser renderer never downloads image binaries; validate retained references before fetching them downstream.
request_timeout_secs	integer	No	7	From 2 to 25 seconds per HTTP request. The global extraction deadline can shorten this budget.
browser_timeout_secs	integer	No	20	From 5 to 60 seconds for Playwright navigation. The global extraction deadline always takes precedence.
overall_timeout_secs	integer	No	20	From 8 to 120 seconds, shared by HTTP, browser, and parsing phases. It starts after the Actor process begins, so it excludes container allocation and cold-start time.
max_content_bytes	integer	No	3000000	From 100,000 to 5,000,000 decoded HTML bytes for the main page. Browser subresources have separate per-request and total budgets.

Unknown input fields and values of the wrong type fail validation instead of being silently ignored.

Execution modes

Configuration	Behavior	Typical reason to use it
render_js: false, browser_fallback: true	Lightweight HTTP first; Playwright is used only after recoverable fetch/extraction failure or insufficient readable content.	Default balance between compute use and compatibility.
render_js: false, browser_fallback: false	Fetch and parse only; Chromium is never imported or launched.	Strict cost and latency control for server-rendered HTML.
render_js: true	Playwright from the start; browser_fallback is irrelevant.	Pages known to require client-side rendering.

Content is considered sufficient when it reaches both min_content_chars and 20 words. A forced-browser or fetch-only run can still succeed below that threshold and reports the condition in warnings. If a browser fallback fails after a usable lightweight candidate was extracted, the Actor returns that lightweight result with a warning.

The default application deadline is 20 seconds. The lightweight request defaults to 7 seconds and reserves time for a possible browser fallback. Container allocation happens before this application timer. Remote smoke tests observed fresh-container runs taking up to 33 seconds end to end, so the separate production gateway in this repository uses a 55-second Actor run timeout and a 60-second upstream deadline. These limited smoke observations are not an SLA, latency benchmark, or cost guarantee.

Runs that force or allow Playwright request 512 MB by default. Explicit browser_fallback: false fetch-only runs request 256 MB.

Output

On success, the Actor writes exactly one item to the default Dataset. The Dataset is the sole output contract; the Actor deliberately avoids a duplicate OUTPUT Key-Value Store copy.

The exact response shape from run-sync-get-dataset-items is a JSON array containing that one item. The values below are illustrative, while every field and nested field shown is part of the runtime contract:

[
  {
    "success": true,
    "title": "Example article",
    "author": "Ada Example",
    "markdown": "## Main section\n\nClean content...",
    "word_count": 1500,
    "estimated_tokens": 2100,
    "url": "https://example.com/article",
    "final_url": "https://www.example.com/article",
    "rendered_with": "fetch",
    "extracted_at": "2026-08-30T12:00:00.000Z",
    "warnings": [],
    "metadata": {
      "strategy": "lightweight",
      "excerpt": "Clean content...",
      "site_name": "Example",
      "language": "en",
      "published_time": null,
      "content_characters": 9200,
      "source_bytes": 48000,
      "http_status": 200,
      "token_estimation_method": "ceil(markdown_characters/4)",
      "timing_ms": {
        "total": 320,
        "fetch": 240,
        "browser": null,
        "parse": 80
      }
    }
  }
]

author, metadata.excerpt, metadata.site_name, metadata.language, and metadata.published_time can be null. rendered_with is either fetch or playwright; metadata.strategy is lightweight, browser-fallback, or browser-forced.

estimated_tokens is a model-independent heuristic (ceil(markdown.length / 4)), not an exact tokenizer result. Callers that require billing-grade counts should tokenize with the target model's tokenizer.

The Actor validates the serialized one-item Dataset response, including JSON escaping and the surrounding array, against the gateway's 1 MiB (1,048,576-byte) upstream limit. Markdown is capped at 960 KiB (983,040 UTF-8 bytes), leaving a documented 64 KiB allowance for URLs, metadata, JSON escaping, and the surrounding array. The complete serialized result is still checked, so an oversized result fails with CONTENT_TOO_LARGE instead of becoming a successful response that the gateway cannot consume.

API integration

Set APIFY_TOKEN to a token that is authorized to run the Actor. The synchronous endpoint returns an array, so each example explicitly reads its first item.

cURL

Requires jq to select and process the result:

curl --fail-with-body --request POST \
  "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items?clean=true&limit=1&timeout=55" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"url":"https://example.com/","render_js":false}' \
  | jq -r '.[0] | "Title: \(.title)\nRenderer: \(.rendered_with)\n\n\(.markdown)"'

limit=1 is the response-item limit. The Actor itself writes exactly one item, so a pay-per-result maxItems query parameter is not needed to shape this response.

Python

Install the HTTP client with python -m pip install requests:

import os

import requests

response = requests.post(
    "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items",
    params={"clean": "true", "limit": 1, "timeout": 55},
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    json={"url": "https://example.com/", "render_js": False},
    timeout=70,
)
response.raise_for_status()

items = response.json()
if not isinstance(items, list):
    raise RuntimeError("Expected the Apify Dataset response to be a list")
if len(items) != 1:
    raise RuntimeError(f"Expected one Dataset item, received {len(items)}")

result = items[0]
print(result["title"])
print(result["markdown"])

Node.js

This example uses the built-in fetch available in current Node.js versions:

const endpoint = new URL(
  "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items",
);
endpoint.search = new URLSearchParams({
  clean: "true",
  limit: "1",
  timeout: "55",
});

const response = await fetch(endpoint, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/",
    render_js: false,
  }),
  signal: AbortSignal.timeout(70_000),
});

if (!response.ok) {
  throw new Error(`Apify ${response.status}: ${await response.text()}`);
}

const items = await response.json();
if (!Array.isArray(items)) {
  throw new Error("Expected the Apify Dataset response to be an array");
}
if (items.length !== 1) {
  throw new Error(`Expected one Dataset item, received ${items.length}`);
}

const [result] = items;
console.log(result.title);
console.log(result.markdown);

For longer-running or disconnected workflows, start the Actor asynchronously with the standard Apify Run Actor endpoint, wait for completion or use a webhook, and then read defaultDatasetId. This Actor's output still remains the same single Dataset item.

Failure semantics

Input, security, HTTP, timeout, size, browser, and extraction failures fail the Actor run and do not push a misleading { "success": false } item. With run-sync-get-dataset-items, success is an HTTP response containing the one-item JSON array above; a failed run is a non-2xx Apify API response and must be handled before parsing Dataset items.

Normalized runtime error codes include INPUT_INVALID, SSRF_BLOCKED, REDIRECT_INVALID, DNS_LOOKUP_FAILED, HTTP_ERROR, REQUEST_FAILED, REQUEST_TIMEOUT, UNSUPPORTED_CONTENT_TYPE, EMPTY_CONTENT, CONTENT_TOO_LARGE, and BROWSER_FAILED. Persisted logs and terminal errors contain normalized codes and error types, never raw target URLs or upstream/Playwright messages.

Warnings are successful-result diagnostics, not failures. They can report validated redirects, blocked browser resources, a browser fallback, short readable content, or a failed browser fallback followed by a usable lightweight result.

Browser safety and resource limits

Playwright is imported and Chromium is launched only in a forced or automatic fallback path. Chromium never receives direct network access for page HTTP traffic: every document, redirect, script, stylesheet, XHR, and fetch request is intercepted, DNS-validated, IP-pinned, downloaded through the safe Node client, and fulfilled back into the page. Service workers and WebSockets are blocked. Non-idempotent browser methods, including POST, as well as popups, downloads, images, media, fonts, event streams, and response cookies are blocked.

Those controls intentionally limit compatibility. Sites that require POST-based GraphQL or data calls, response-cookie sessions, authentication, or similar state may not hydrate even with render_js: true. The option means JavaScript execution is forced; it does not relax the network safety policy.

The browser path allows at most 120 HTTP requests, 8 simultaneous downloads, 2 MB per subresource, max_content_bytes for a document, and at most 15 MB total or three times the configured main-page limit, whichever is lower. The lightweight path applies max_content_bytes cumulatively across every redirect response and the final response, rather than granting a fresh allowance per hop. The HTML default is 3 MB and is configurable up to 5 MB.

A linear preflight rejects source markup with more than 50,000 approximate opening elements, and Playwright checks the live DOM before serializing it. These limits deliberately trade some compatibility for bounded memory and SSRF resistance. JSDOM and Readability parsing are synchronous and cannot be interrupted mid-operation; byte and element caps plus deadline checkpoints bound the practical risk but do not make parsing fully preemptible.

When links or images are enabled, their protocol and obvious URL shape are filtered, but public-looking hostnames inside extracted content are not DNS-resolved. Treat retained references as untrusted data and validate them before any downstream fetch.

Responsible use

Use the Actor only for pages you are authorized to access and process. Respect site terms, copyright, privacy, rate limits, and applicable law. robots.txt is a crawler convention and can be context-dependent; this single-page Actor does not automatically fetch or interpret it. Check it yourself when your use case requires it.

Local development

Node.js 24.15 or newer is required.

npm ci
npm run check
npm run build

For a local Actor run, place valid JSON at storage/key_value_stores/default/INPUT.json and run npx apify-cli run. Networked runs should target only a page you control or are allowed to scrape. Unit tests use inline HTML and injected DNS resolvers; they do not access the network.

The production Docker image pins the official apify/actor-node-playwright-chrome:24-1.62.1 base in both stages and pins the matching Playwright npm package to 1.62.1. Forced-browser and automatic-fallback paths were smoke-tested remotely at 512 MB before making it the default allocation. These tests verify the execution paths, not universal site compatibility.