# LLM Reader & Web2Markdown (`dekaz/llm-reader-web2markdown`) Actor

Convert one public web page or JavaScript-rendered article into clean Markdown for LLM, RAG, and AI-agent pipelines using lightweight HTTP fetching with an SSRF-hardened Playwright fallback.

- **URL**: https://apify.com/dekaz/llm-reader-web2markdown.md
- **Developed by:** [Progamadores.com](https://apify.com/dekaz) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.50 / 1,000 web page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## LLM Reader & Web2Markdown

Convert one public web page into compact, main-content Markdown for LLMs, RAG ingestion, AI agents, and downstream content pipelines. This web page-to-Markdown API combines a lightweight HTTP fetch with an SSRF-hardened Playwright fallback for JavaScript-rendered pages.

The Actor uses Mozilla Readability to identify the primary article content, filters common page furniture, and uses Turndown to produce consistent Markdown plus structured source metadata and an approximate token count. It processes exactly one URL per run; it is an article content extractor and HTML-to-Markdown scraper, not a whole-site crawler.

### Common use cases

- Prepare a public article or documentation page for chunking, embeddings, and a RAG knowledge base.
- Give an AI agent clean page text, title, author, source URL, and extraction metadata without passing raw HTML.
- Normalize blog posts, help-center articles, or public reports before summarization or classification.
- Extract readable content from a JavaScript-rendered page when the site's network behavior is compatible with the Actor's safety policy.
- Feed a downstream content-monitoring or diff pipeline with repeatable Markdown snapshots, one URL at a time.

The Actor does not crawl links, discover sitemaps, process PDFs, or bypass authentication, CAPTCHAs, paywalls, access controls, or anti-bot protections.

### Input

A minimal run only needs a public URL:

```json
{
  "url": "https://example.com/article"
}
```

All supported options:

```json
{
  "url": "https://example.com/article",
  "render_js": false,
  "browser_fallback": true,
  "min_content_chars": 500,
  "include_links": false,
  "include_images": false,
  "request_timeout_secs": 7,
  "browser_timeout_secs": 20,
  "overall_timeout_secs": 20,
  "max_content_bytes": 3000000
}
```

| Field | Type | Required | Default | Allowed values and behavior |
| --- | --- | --- | --- | --- |
| `url` | string | Yes | — | Absolute public `http://` or `https://` URL, at most 2,048 characters; URL fragments are discarded. URL credentials, local hostnames, private/link-local/loopback/special IP addresses, mixed public-private DNS answers, nonstandard ports, and unsafe redirects are rejected. Only ports 80 and 443 are supported. |
| `render_js` | boolean | No | `false` | When `true`, launch Playwright immediately. When `false`, try lightweight HTTP extraction first. |
| `browser_fallback` | boolean | No | `true` | When `render_js` is `false`, allow Playwright if lightweight extraction fails recoverably or is too short. Set to `false` for a strict fetch-only run. |
| `min_content_chars` | integer | No | `500` | From 100 to 10,000. Lightweight content below this text-character threshold triggers browser fallback when enabled. Content must also contain at least 20 words to be considered sufficient. |
| `include_links` | boolean | No | `false` | Keep filtered HTTP(S) and `mailto:` references. Relative links are resolved, but retained public-looking hostnames are not DNS-validated; validate them before any downstream fetch. |
| `include_images` | boolean | No | `false` | Keep filtered image references in Markdown. The browser renderer never downloads image binaries; validate retained references before fetching them downstream. |
| `request_timeout_secs` | integer | No | `7` | From 2 to 25 seconds per HTTP request. The global extraction deadline can shorten this budget. |
| `browser_timeout_secs` | integer | No | `20` | From 5 to 60 seconds for Playwright navigation. The global extraction deadline always takes precedence. |
| `overall_timeout_secs` | integer | No | `20` | From 8 to 120 seconds, shared by HTTP, browser, and parsing phases. It starts after the Actor process begins, so it excludes container allocation and cold-start time. |
| `max_content_bytes` | integer | No | `3000000` | From 100,000 to 5,000,000 decoded HTML bytes for the main page. Browser subresources have separate per-request and total budgets. |

Unknown input fields and values of the wrong type fail validation instead of being silently ignored.

### Execution modes

| Configuration | Behavior | Typical reason to use it |
| --- | --- | --- |
| `render_js: false`, `browser_fallback: true` | Lightweight HTTP first; Playwright is used only after recoverable fetch/extraction failure or insufficient readable content. | Default balance between compute use and compatibility. |
| `render_js: false`, `browser_fallback: false` | Fetch and parse only; Chromium is never imported or launched. | Strict cost and latency control for server-rendered HTML. |
| `render_js: true` | Playwright from the start; `browser_fallback` is irrelevant. | Pages known to require client-side rendering. |

Content is considered sufficient when it reaches both `min_content_chars` and 20 words. A forced-browser or fetch-only run can still succeed below that threshold and reports the condition in `warnings`. If a browser fallback fails after a usable lightweight candidate was extracted, the Actor returns that lightweight result with a warning.

The default application deadline is 20 seconds. The lightweight request defaults to 7 seconds and reserves time for a possible browser fallback. Container allocation happens before this application timer. Remote smoke tests observed fresh-container runs taking up to 33 seconds end to end, so the separate production gateway in this repository uses a 55-second Actor run timeout and a 60-second upstream deadline. These limited smoke observations are not an SLA, latency benchmark, or cost guarantee.

Runs that force or allow Playwright request 512 MB by default. Explicit `browser_fallback: false` fetch-only runs request 256 MB.

### Output

On success, the Actor writes exactly one item to the default Dataset. The Dataset is the sole output contract; the Actor deliberately avoids a duplicate `OUTPUT` Key-Value Store copy.

The exact response shape from `run-sync-get-dataset-items` is a JSON array containing that one item. The values below are illustrative, while every field and nested field shown is part of the runtime contract:

```json
[
  {
    "success": true,
    "title": "Example article",
    "author": "Ada Example",
    "markdown": "## Main section\n\nClean content...",
    "word_count": 1500,
    "estimated_tokens": 2100,
    "url": "https://example.com/article",
    "final_url": "https://www.example.com/article",
    "rendered_with": "fetch",
    "extracted_at": "2026-08-30T12:00:00.000Z",
    "warnings": [],
    "metadata": {
      "strategy": "lightweight",
      "excerpt": "Clean content...",
      "site_name": "Example",
      "language": "en",
      "published_time": null,
      "content_characters": 9200,
      "source_bytes": 48000,
      "http_status": 200,
      "token_estimation_method": "ceil(markdown_characters/4)",
      "timing_ms": {
        "total": 320,
        "fetch": 240,
        "browser": null,
        "parse": 80
      }
    }
  }
]
```

`author`, `metadata.excerpt`, `metadata.site_name`, `metadata.language`, and `metadata.published_time` can be `null`. `rendered_with` is either `fetch` or `playwright`; `metadata.strategy` is `lightweight`, `browser-fallback`, or `browser-forced`.

`estimated_tokens` is a model-independent heuristic (`ceil(markdown.length / 4)`), not an exact tokenizer result. Callers that require billing-grade counts should tokenize with the target model's tokenizer.

The Actor validates the serialized one-item Dataset response, including JSON escaping and the surrounding array, against the gateway's 1 MiB (1,048,576-byte) upstream limit. Markdown is capped at 960 KiB (983,040 UTF-8 bytes), leaving a documented 64 KiB allowance for URLs, metadata, JSON escaping, and the surrounding array. The complete serialized result is still checked, so an oversized result fails with `CONTENT_TOO_LARGE` instead of becoming a successful response that the gateway cannot consume.

### API integration

Set `APIFY_TOKEN` to a token that is authorized to run the Actor. The synchronous endpoint returns an array, so each example explicitly reads its first item.

#### cURL

Requires `jq` to select and process the result:

```bash
curl --fail-with-body --request POST \
  "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items?clean=true&limit=1&timeout=55" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"url":"https://example.com/","render_js":false}' \
  | jq -r '.[0] | "Title: \(.title)\nRenderer: \(.rendered_with)\n\n\(.markdown)"'
```

`limit=1` is the response-item limit. The Actor itself writes exactly one item, so a pay-per-result `maxItems` query parameter is not needed to shape this response.

#### Python

Install the HTTP client with `python -m pip install requests`:

```python
import os

import requests

response = requests.post(
    "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items",
    params={"clean": "true", "limit": 1, "timeout": 55},
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    json={"url": "https://example.com/", "render_js": False},
    timeout=70,
)
response.raise_for_status()

items = response.json()
if not isinstance(items, list):
    raise RuntimeError("Expected the Apify Dataset response to be a list")
if len(items) != 1:
    raise RuntimeError(f"Expected one Dataset item, received {len(items)}")

result = items[0]
print(result["title"])
print(result["markdown"])
```

#### Node.js

This example uses the built-in `fetch` available in current Node.js versions:

```javascript
const endpoint = new URL(
  "https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/run-sync-get-dataset-items",
);
endpoint.search = new URLSearchParams({
  clean: "true",
  limit: "1",
  timeout: "55",
});

const response = await fetch(endpoint, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/",
    render_js: false,
  }),
  signal: AbortSignal.timeout(70_000),
});

if (!response.ok) {
  throw new Error(`Apify ${response.status}: ${await response.text()}`);
}

const items = await response.json();
if (!Array.isArray(items)) {
  throw new Error("Expected the Apify Dataset response to be an array");
}
if (items.length !== 1) {
  throw new Error(`Expected one Dataset item, received ${items.length}`);
}

const [result] = items;
console.log(result.title);
console.log(result.markdown);
```

For longer-running or disconnected workflows, start the Actor asynchronously with the standard Apify Run Actor endpoint, wait for completion or use a webhook, and then read `defaultDatasetId`. This Actor's output still remains the same single Dataset item.

### Failure semantics

Input, security, HTTP, timeout, size, browser, and extraction failures fail the Actor run and do not push a misleading `{ "success": false }` item. With `run-sync-get-dataset-items`, success is an HTTP response containing the one-item JSON array above; a failed run is a non-2xx Apify API response and must be handled before parsing Dataset items.

Normalized runtime error codes include `INPUT_INVALID`, `SSRF_BLOCKED`, `REDIRECT_INVALID`, `DNS_LOOKUP_FAILED`, `HTTP_ERROR`, `REQUEST_FAILED`, `REQUEST_TIMEOUT`, `UNSUPPORTED_CONTENT_TYPE`, `EMPTY_CONTENT`, `CONTENT_TOO_LARGE`, and `BROWSER_FAILED`. Persisted logs and terminal errors contain normalized codes and error types, never raw target URLs or upstream/Playwright messages.

Warnings are successful-result diagnostics, not failures. They can report validated redirects, blocked browser resources, a browser fallback, short readable content, or a failed browser fallback followed by a usable lightweight result.

### Browser safety and resource limits

Playwright is imported and Chromium is launched only in a forced or automatic fallback path. Chromium never receives direct network access for page HTTP traffic: every document, redirect, script, stylesheet, XHR, and fetch request is intercepted, DNS-validated, IP-pinned, downloaded through the safe Node client, and fulfilled back into the page. Service workers and WebSockets are blocked. Non-idempotent browser methods, including POST, as well as popups, downloads, images, media, fonts, event streams, and response cookies are blocked.

Those controls intentionally limit compatibility. Sites that require POST-based GraphQL or data calls, response-cookie sessions, authentication, or similar state may not hydrate even with `render_js: true`. The option means JavaScript execution is forced; it does not relax the network safety policy.

The browser path allows at most 120 HTTP requests, 8 simultaneous downloads, 2 MB per subresource, `max_content_bytes` for a document, and at most 15 MB total or three times the configured main-page limit, whichever is lower. The lightweight path applies `max_content_bytes` cumulatively across every redirect response and the final response, rather than granting a fresh allowance per hop. The HTML default is 3 MB and is configurable up to 5 MB.

A linear preflight rejects source markup with more than 50,000 approximate opening elements, and Playwright checks the live DOM before serializing it. These limits deliberately trade some compatibility for bounded memory and SSRF resistance. JSDOM and Readability parsing are synchronous and cannot be interrupted mid-operation; byte and element caps plus deadline checkpoints bound the practical risk but do not make parsing fully preemptible.

When links or images are enabled, their protocol and obvious URL shape are filtered, but public-looking hostnames inside extracted content are not DNS-resolved. Treat retained references as untrusted data and validate them before any downstream fetch.

### Responsible use

Use the Actor only for pages you are authorized to access and process. Respect site terms, copyright, privacy, rate limits, and applicable law. `robots.txt` is a crawler convention and can be context-dependent; this single-page Actor does not automatically fetch or interpret it. Check it yourself when your use case requires it.

### Local development

Node.js 24.15 or newer is required.

```bash
npm ci
npm run check
npm run build
```

For a local Actor run, place valid JSON at `storage/key_value_stores/default/INPUT.json` and run `npx apify-cli run`. Networked runs should target only a page you control or are allowed to scrape. Unit tests use inline HTML and injected DNS resolvers; they do not access the network.

The production Docker image pins the official `apify/actor-node-playwright-chrome:24-1.62.1` base in both stages and pins the matching Playwright npm package to `1.62.1`. Forced-browser and automatic-fallback paths were smoke-tested remotely at 512 MB before making it the default allocation. These tests verify the execution paths, not universal site compatibility.

# Actor input Schema

## `url` (type: `string`):

Absolute public http:// or https:// URL. Credentials, private/special IP ranges, local hostnames, and ports other than 80/443 are rejected.

## `render_js` (type: `boolean`):

When true, launch Playwright immediately. When false, use lightweight HTTP first and fall back to Playwright only if browser\_fallback is enabled and needed.

## `browser_fallback` (type: `boolean`):

Direct Actor option: when render\_js is false, allow Playwright only if lightweight extraction fails or is too short. Set false for a strict low-cost, fetch-only run.

## `min_content_chars` (type: `integer`):

Lightweight results shorter than this trigger browser fallback. This threshold does not pad or fabricate content.

## `include_links` (type: `boolean`):

Opt in to protocol/shape-filtered but untrusted HTTP(S) and mailto references. Relative links are resolved; public-looking hostnames are not DNS-validated, so downstream consumers must validate before fetching.

## `include_images` (type: `boolean`):

Opt in to protocol/shape-filtered but untrusted image references. Image binaries are never downloaded by the browser renderer; downstream consumers must validate references before fetching.

## `request_timeout_secs` (type: `integer`):

Per-request timeout. The global extraction deadline can shorten it, especially before a browser fallback.

## `browser_timeout_secs` (type: `integer`):

Maximum Playwright navigation budget. The global extraction deadline always takes precedence.

## `overall_timeout_secs` (type: `integer`):

Application deadline shared by HTTP, browser, and parsing phases. It starts after the Actor process begins and excludes container allocation and cold-start time; surrounding callers must configure a larger end-to-end deadline.

## `max_content_bytes` (type: `integer`):

Maximum decoded HTML bytes for the main page. Browser subresources also have per-request and total budgets.

## Actor input object example

```json
{
  "url": "https://example.com/",
  "render_js": false,
  "browser_fallback": true,
  "min_content_chars": 500,
  "include_links": false,
  "include_images": false,
  "request_timeout_secs": 7,
  "browser_timeout_secs": 20,
  "overall_timeout_secs": 20,
  "max_content_bytes": 3000000
}
```

# Actor output Schema

## `results` (type: `string`):

Default Dataset containing exactly one item on success. Suitable for run-sync-get-dataset-items.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://example.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("dekaz/llm-reader-web2markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "url": "https://example.com/" }

# Run the Actor and wait for it to finish
run = client.actor("dekaz/llm-reader-web2markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://example.com/"
}' |
apify call dekaz/llm-reader-web2markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,dekaz/llm-reader-web2markdown"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/O0BUo5Aeddo3ealQ6/builds/gyUyte7z6cPU75K6n/openapi.json