# Website to Markdown for AI & LLMs (RAG-ready) (`tidelane/website-to-markdown`) Actor

Turn any web page into clean Markdown for ChatGPT, Claude, RAG pipelines and AI agents. Real browser renders JavaScript sites, the main content is extracted without menus, ads and cookie banners, and tables, code and links are kept. Token estimate included.

- **URL**: https://apify.com/tidelane/website-to-markdown.md
- **Developed by:** [Tidelane Data](https://apify.com/tidelane) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.80 / 1,000 page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website to Markdown for AI & LLMs (RAG-ready)

Turn web pages into clean Markdown that ChatGPT, Claude, Gemini, RAG pipelines and AI agents can read. A real browser renders the page (JavaScript sites work), the main content is extracted with Mozilla Readability (the engine behind Firefox Reader View), and menus, ads, footers and cookie banners are dropped. Headings, tables, code blocks, lists and links stay.

- ✅ **Clean main content** or the **whole page** without navigation and footer
- ✅ **GitHub-flavored Markdown**: tables, fenced code, lists, absolute links
- ✅ **Token estimate** per page (`approxTokens`), word and character counts, optional length limit
- ✅ **Metadata**: title, description, site name, language, publish date
- ✅ **Works on modern sites**: JavaScript rendering, strict security policies and AMD/RequireJS pages
- ✅ **Fair billing**: failed pages, error pages (4xx/5xx) and pages excluded by robots.txt are free

### What can you use it for?

- **RAG and knowledge bases:** feed documentation, help centers and articles into a vector database
- **AI agents:** let an agent read any page through the Apify API or MCP and get compact text instead of raw HTML
- **Summaries and research:** collect articles for summarization, translation or analysis
- **Content migration:** move pages from an old CMS into Markdown-based docs

### Input example

```json
{
    "urls": ["https://github.com/mozilla/readability", "https://en.wikipedia.org/wiki/Markdown"],
    "mode": "article",
    "includeLinks": true,
    "includeImages": false
}
```

### Output example

Real output from a test run (Markdown shortened):

```json
{
    "url": "https://github.com/mozilla/readability",
    "finalUrl": "https://github.com/mozilla/readability",
    "statusCode": 200,
    "title": "GitHub - mozilla/readability: A standalone version of the readability lib",
    "description": "A standalone version of the readability lib. Contribute to mozilla/readability development by creating an account on GitHub.",
    "siteName": "GitHub",
    "language": "en",
    "publishedTime": null,
    "mode": "article",
    "markdown": "A standalone version of the readability library used for [Firefox Reader View](https://support.mozilla.org/kb/firefox-reader-view-clutter-free-web-pages).\n\n## Installation\n\nReadability is available on npm: ...",
    "wordCount": 1025,
    "charCount": 7300,
    "approxTokens": 1825,
    "truncated": false,
    "convertedAt": "2026-10-20T21:19:49.560Z"
}
```

`mode` tells you what was used: `article` (main content) or `full` (whole page, used automatically when a page has no clear main text, like home pages and link lists). A summary per URL is saved as `OUTPUT` in the key-value store.

### Pricing

A fixed price per converted page. In our tests 6 pages took about a minute with the default 2 GB memory.

Paid Apify plans pay less: 10% off on Starter, 20% on Scale and 30% on Business and higher plans.

### FAQ

**Does it crawl a whole website?** No, every URL is one page. To convert a whole site, pass its page URLs (for example from its sitemap).

**Why is a page "blocked_by_robots"?** By default the Actor respects robots.txt. Turn off **Respect robots.txt** only for your own sites or with the owner's permission.

**Is it legal?** Converting public pages for reading, research and AI processing is common. Respect copyright and the site's terms when you store or republish content. You are responsible for your use of the data.

***

#### Impressum

Levi Griesinger, Tidelane Data\
Besigheimer Straße 18\
74369, Löchgau, Deutschland\
E-Mail: LeviGriesinger@gmail.com

# Actor input Schema

## `urls` (type: `array`):

Pages to convert, one per line. Each URL is one page (no crawling).

## `mode` (type: `string`):

Main content uses Mozilla Readability, like Firefox Reader View. Pages without a clear main text fall back to the whole page.

## `includeLinks` (type: `boolean`):

Keep hyperlinks as Markdown links with absolute URLs. Turn off for plain text with fewer tokens.

## `includeImages` (type: `boolean`):

Include images as Markdown image links.

## `maxLength` (type: `integer`):

Cut the Markdown after this many characters (0 = no limit). About 4 characters are one token.

## `waitMs` (type: `integer`):

Wait after the page has loaded, for content that appears late.

## `respectRobotsTxt` (type: `boolean`):

Skip pages whose robots.txt forbids automated access. Turn off only for your own sites or with the owner's permission.

## `timeoutSecs` (type: `integer`):

Pages that take longer fail and are not charged.

## `maxConcurrency` (type: `integer`):

More is faster but needs more memory.

## Actor input object example

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Markdown",
    "https://docs.apify.com/platform/actors"
  ],
  "mode": "article",
  "includeLinks": true,
  "includeImages": false,
  "maxLength": 0,
  "waitMs": 1000,
  "respectRobotsTxt": true,
  "timeoutSecs": 45,
  "maxConcurrency": 3
}
```

# Actor output Schema

## `results` (type: `string`):

All results of the run as dataset items.

## `summary` (type: `string`):

Summary per input with status (ok, error and the reason).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://en.wikipedia.org/wiki/Markdown",
        "https://docs.apify.com/platform/actors"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidelane/website-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://en.wikipedia.org/wiki/Markdown",
        "https://docs.apify.com/platform/actors",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("tidelane/website-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://en.wikipedia.org/wiki/Markdown",
    "https://docs.apify.com/platform/actors"
  ]
}' |
apify call tidelane/website-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidelane/website-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bXk2wliJCl98kaUGU/builds/QTpl2afly7ZHUfSo3/openapi.json