# Website to Markdown Crawler for AI, LLM & RAG - $1/1K (`plotpointtech/website-to-markdown-crawler`) Actor

Crawl any website and get clean Markdown of every page for LLMs, RAG, vector databases and chatbots. Menus, footers, ads and cookie banners removed; headings, tables, code and links kept. $1 per 1,000 pages.

- **URL**: https://apify.com/plotpointtech/website-to-markdown-crawler.md
- **Developed by:** [Plot Point Technologies](https://apify.com/plotpointtech) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.62 / 1,000 page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website to Markdown Crawler for AI, LLMs and RAG

**Turn any public website into clean Markdown that is ready for large language models, retrieval-augmented generation (RAG), vector databases, chatbots and search indexes.** Give it one or more start URLs; it saves the main content of every page as Markdown, with menus, cookie banners, footers, sidebars, ads and scripts removed, and headings, paragraphs, lists, tables, code blocks and links kept. **$1 per 1,000 pages fetched directly; $3 per 1,000 pages that need a real browser.**

### What can this website to Markdown crawler do?

- **Convert single pages or crawl whole sites** (same website only), with page and depth limits.
- **Find pages three ways:** links on the pages, the site's **sitemap.xml** (including sitemap indexes and `.xml.gz`), and the site's **llms.txt** file (including the Markdown files it lists). You can also enter a sitemap or llms.txt URL directly as a start URL.
- **Stay inside a section:** starting at `/docs/` only crawls pages under `/docs/`.
- **Include or skip URLs** with glob patterns such as `https://example.com/docs/**`, `*/blog/*` or plain text such as `/tag/`.
- **Remove boilerplate automatically.** Three extraction modes: *Main content* (default: keeps the main area, removes menus, sidebars, footers, banners), *Article text* (Readability-style extraction with trafilatura, for blogs and news), and *Full page*.
- **Control it with CSS selectors:** remove elements (navigation, footers, dialogs and common cookie banners by default) or keep only specific elements such as `article` or `.docs-content`.
- **Handle JavaScript sites automatically.** In *Auto* mode every page is first fetched directly (fast and cheap); pages that come back empty because they are built with JavaScript are opened in a real headless browser. If most pages of a site need the browser, the crawler goes browser-first for that site. *Always* forces the browser for every page.
- **Convert linked PDF and Word (DOCX) files** to Markdown and text, with headings, lists and tables, plus document metadata (optional).
- **Chunk for RAG:** optionally split every page into chunks of a chosen size and overlap on heading, paragraph and sentence boundaries, each with its heading path.
- **Skip duplicates:** pages with identical text, or whose canonical URL points to a page already saved, are not saved or charged.
- **Crawl incrementally:** only fetch sitemap entries changed since a date, and/or remember page hashes in a named key-value store so the next run saves only new and changed pages.
- **Rich metadata:** title, description, canonical URL, language, author, published and modified dates, keywords, Open Graph tags and useful HTTP headers (Last-Modified, ETag, Content-Type and more).
- **Skip thin pages** below a word count (not charged).
- **Report AI-usage preferences:** Content-Signal rules a site declares in robots.txt (for example `ai-train=no`) are returned with every page so your pipeline can honour them.

### What data do you get?

| Field | Description |
|---|---|
| `url`, `loadedUrl` | Requested URL and final URL after redirects |
| `title`, `description` | Page title and meta description |
| `markdown` | Clean Markdown (headings, lists, tables, links, code blocks) |
| `text` | Plain text (Markdown formatting removed); can be turned off |
| `html` | Cleaned main-content HTML (only with **Save cleaned HTML**) |
| `chunks` | RAG chunks: `index`, `text`, `headings` (section path), `charCount` (only with **Chunk size** above 0) |
| `wordCount`, `language` | Size and language of the content |
| `contentType` | `html`, `markdown`, `pdf` or `docx` |
| `source`, `referrerUrl`, `depth` | How the URL was found (`start`, `link`, `sitemap`, `llms.txt`), the page that linked to it, and crawl depth |
| `httpStatus`, `canonicalUrl` | Status code and canonical link |
| `renderedWithBrowser` | Whether a browser was needed |
| `metadata` | `title`, `description`, `canonicalUrl`, `languageCode`, `author`, `keywords`, `publishedAt`, `modifiedAt`, `siteName`, `image`, `openGraph` (all `og:` tags), `headers` (Content-Type, Last-Modified, ETag, Date, Cache-Control, X-Robots-Tag); for documents also `pageCount` and `fileSizeBytes` |
| `contentHash` | Hash of the page text, for change detection and de-duplication |
| `changeStatus` | `new` or `changed` (only in incremental mode) |
| `crawledAt` | When the page was fetched |
| `contentSignals` | Site's declared Content-Signal preferences, or `null` |

#### Main input options

| Option | Default | What it does |
|---|---|---|
| Start URLs | Python tutorial | Pages, domains, sitemap URLs, llms.txt URLs, or PDF/DOCX links |
| Follow links / Maximum pages / Maximum link depth | on / 20 / 2 | Crawl scope |
| Stay within the start URL's folder | on | Only crawl under the start path |
| Only crawl / Skip URLs matching | none | Glob patterns (`*` matches anything, including `/`) |
| Find pages in sitemap.xml / llms.txt | off / off | Extra URL sources |
| Render JavaScript | Auto | Auto, Always (browser for every page) or Never |
| Content extraction | Main content | Main content, Article text (trafilatura) or Full page |
| Remove elements / Keep only elements | nav, footer, aside, dialogs, cookie banners / none | CSS selectors |
| Save Markdown / plain text / cleaned HTML | on / on / off | Output formats |
| Chunk size / overlap | 0 (off) / 150 | RAG chunking, in characters |
| Convert linked PDF and DOCX files | off | Max file size 20 MB, first 300 PDF pages |
| Skip duplicate pages | on | By text hash and canonical URL |
| Only pages modified since | none | Date filter for sitemap entries and page dates |
| Remember pages in key-value store | none | Incremental mode: save only new and changed pages |

### Who uses it?

- **AI and chatbot builders** loading documentation, help centers or websites into a knowledge base.
- **Developers** feeding docs sites to coding assistants and agents.
- **Content and SEO teams** exporting a site's copy for review, translation or migration.
- **Researchers** collecting readable article text from lists of URLs.

### How to use it

1. Click **Try for free** (or **Start**).
2. Enter **Start URLs** (`example.com` works).
3. Keep **Follow links on the same website** on to crawl, and set **Maximum pages** and **Maximum link depth**.
4. Run it and download JSON, JSONL, CSV or Excel, or read the dataset from your RAG pipeline through the API.

### Input example

```json
{
  "startUrls": [{ "url": "https://docs.python.org/3/tutorial/index.html" }],
  "crawlLinks": true,
  "maxPages": 200,
  "maxDepth": 3,
  "useSitemaps": true,
  "excludeUrlPatterns": ["**/genindex*", "**/search.html*"],
  "renderJavaScript": "auto",
  "contentExtraction": "smart",
  "chunkSize": 1500,
  "chunkOverlap": 150,
  "downloadFiles": true,
  "incrementalStoreName": "python-docs-state",
  "minWordCount": 50
}
```

### Output example

```json
{
  "url": "https://docs.python.org/3/tutorial/introduction.html",
  "loadedUrl": "https://docs.python.org/3/tutorial/introduction.html",
  "title": "3. An Informal Introduction to Python — Python 3.15.2 documentation",
  "description": "In the following examples, input and output are distinguished by the presence or absence of prompts...",
  "language": "en",
  "wordCount": 2735,
  "depth": 1,
  "httpStatus": 200,
  "canonicalUrl": "https://docs.python.org/3/tutorial/introduction.html",
  "renderedWithBrowser": false,
  "contentType": "html",
  "source": "link",
  "referrerUrl": "https://docs.python.org/3/tutorial/index.html",
  "crawledAt": "2026-10-28T14:49:00+00:00",
  "markdown": "# 3. An Informal Introduction to Python\n\nIn the following examples, input and output are distinguished by the presence or absence of prompts...",
  "text": "3. An Informal Introduction to Python\n\nIn the following examples, input and output are distinguished...",
  "chunks": [
    { "index": 0, "text": "# 3. An Informal Introduction to Python\n\nIn the following examples...", "headings": ["3. An Informal Introduction to Python"], "charCount": 1432 }
  ],
  "metadata": {
    "title": "3. An Informal Introduction to Python — Python 3.15.2 documentation",
    "canonicalUrl": "https://docs.python.org/3/tutorial/introduction.html",
    "languageCode": "en",
    "author": null,
    "publishedAt": null,
    "modifiedAt": null,
    "siteName": "Python documentation",
    "openGraph": { "title": "3. An Informal Introduction to Python", "type": "website", "site_name": "Python documentation" },
    "headers": { "content-type": "text/html", "last-modified": "Tue, 27 Oct 2026 22:10:41 GMT" }
  },
  "contentHash": "5b0f6c3e8f1a4d2c9e7b6a5d4c3b2a19",
  "contentSignals": null
}
```

### Use it with LangChain, LlamaIndex and vector databases

Every record is one page (or one PDF/DOCX file) with its `url` for citations, so it maps directly onto a "document" in RAG frameworks. Turn on **Chunk size** to get ready-made chunks with their heading path, or chunk the `markdown` field yourself.

**LangChain (Python).** Install `langchain-apify` and load the results as documents:

```python
from langchain_apify import ApifyWrapper
from langchain_core.documents import Document

apify = ApifyWrapper()  # reads APIFY_API_TOKEN from the environment
loader = apify.call_actor(
    actor_id="plotpointtech/website-to-markdown-crawler",
    run_input={"startUrls": [{"url": "https://docs.python.org/3/tutorial/"}], "maxPages": 100},
    dataset_mapping_function=lambda item: Document(
        page_content=item.get("markdown") or "",
        metadata={"source": item["url"], "title": item.get("title")},
    ),
)
docs = loader.load()
```

To use the crawler's own chunks instead, read the dataset with `apify-client` and create one `Document` per entry in `item["chunks"]`, putting `item["url"]` and `chunk["headings"]` in the metadata.

**LlamaIndex (Python).** Use the Apify reader from `llama-index-readers-apify`:

```python
from llama_index.core import Document, VectorStoreIndex
from llama_index.readers.apify import ApifyActor

reader = ApifyActor("YOUR_API_TOKEN")
documents = reader.load_data(
    actor_id="plotpointtech/website-to-markdown-crawler",
    run_input={"startUrls": [{"url": "https://docs.python.org/3/tutorial/"}], "maxPages": 100},
    dataset_mapping_function=lambda item: Document(text=item.get("markdown") or "", metadata={"url": item["url"]}),
)
index = VectorStoreIndex.from_documents(documents)
```

**Vector databases (Pinecone, Qdrant, Weaviate, Chroma, pgvector, OpenAI vector stores).** Run with **Chunk size** set (1,000 to 2,000 characters suits most embedding models), then for each item and chunk embed `chunk["text"]` and upsert it with an ID such as `contentHash + "-" + chunk["index"]` and metadata `url`, `title` and `headings`. Apify's Pinecone and Qdrant integrations can also take the dataset directly after each run.

**Keeping an index fresh.** Schedule the crawler with **Remember pages in key-value store** set to a fixed name. Each run then outputs only pages that are `new` or `changed` (`changeStatus`), so you re-embed just those; unchanged pages are not charged. Use **Only pages modified since** to skip old sitemap entries entirely on large sites.

### How much does it cost?

**$1 per 1,000 pages** ($0.001 per page or document saved), plus a $0.00005 start fee per run. Pages that need a real browser because they are built with JavaScript cost an extra **$0.002** each ($3 per 1,000 in total), because a browser takes many times more computing power than a direct download. In *Auto* mode the browser is only used when a page needs it, so ordinary sites cost $1 per 1,000. Apify's higher subscription plans get lower per-page prices automatically.

| You convert | You pay |
|---|---|
| A 20-page docs section | $0.02 |
| A 500-page help center | $0.50 |
| 10,000 pages | $10.00 |
| 1,000 pages of a JavaScript-only site | $3.00 |

Each converted PDF or DOCX file counts as one page, however many pages it has. Pages that fail to load, are blocked by robots.txt, are not HTML, are duplicates, are unchanged since the last incremental run, or fall below your minimum word count are not saved and not charged.

### Use it via API, Make, Zapier, n8n or an AI agent

**Apify API.** Start a run and get the results back in one HTTP request (replace `YOUR_API_TOKEN` with the token from Apify Console > Settings > Integrations):

```bash
curl -X POST "https://api.apify.com/v2/acts/plotpointtech~website-to-markdown-crawler/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": [{"url": "https://docs.python.org/3/tutorial/index.html"}], "crawlLinks": true, "maxPages": 20, "maxDepth": 2, "minWordCount": 50}'
```

For longer jobs, start the run asynchronously and read the dataset when it finishes. Official `apify-client` libraries exist for Python and JavaScript; the **API** tab on this page has ready-made code.

**No-code automation.** Connect the tool to **Make, Zapier, n8n, Google Sheets, Slack, Airtable** or any webhook with Apify integrations, and use **Schedules** in Apify Console to run it hourly, daily or weekly.

**AI agents (MCP).** Give Claude, ChatGPT, Cursor, VS Code or any MCP-compatible agent direct access to this tool through the hosted Apify MCP server. Add this server URL to your client:

```
https://mcp.apify.com?tools=plotpointtech/website-to-markdown-crawler
```

The agent can then call the tool with the same input fields shown above and read the results.

### FAQ

**Is it legal to crawl websites for AI?**
Collecting public web pages is generally lawful, but rights in the content stay with the website owner. This crawler always respects robots.txt and Crawl-delay, identifies itself as `PlotPointBot`, and returns each site's declared AI-usage preferences in `contentSignals`. Make sure you have the right to use the content for your purpose, and honour those preferences.

**How do I load the output into LangChain, LlamaIndex or a vector database?**
Read the dataset through the Apify API (or the Apify LangChain/LlamaIndex loaders) and chunk the `markdown` field. Each record keeps its `url` for citations.

**Does it include images?**
No. Images are left out of the Markdown because their alt text is not reliable enough for AI use.

**Does it follow subdomains?**
`www.example.com` and `example.com` are treated as the same site; other subdomains are not followed.

**Will it work on sites behind a login or bot check?**
No. Only public pages are supported, and blocking is not bypassed.

### Related tools from Plot Point Technologies

- [AI Web Scraper - Extract Any Website to JSON in Plain English](https://apify.com/plotpointtech/ai-page-extractor): describe the fields you want and get JSON from any web page.
- [PDF to Text & Tables Extractor - Markdown + OCR, $0.40/1K](https://apify.com/plotpointtech/pdf-text-markdown-tables-extractor): PDFs to text, Markdown and CSV tables, with OCR for scans.
- [SEO Audit Crawler - Titles, Meta, H1, Broken Pages & Links](https://apify.com/plotpointtech/sitemap-seo-page-audit): on-page SEO check of every page in a website's sitemap.
- [Wikipedia Scraper - Wikidata Facts, Summaries & Images](https://apify.com/plotpointtech/wikipedia-wikidata-entity-extractor): Wikipedia summaries and Wikidata facts for any entity list.

Plot Point Technologies is not affiliated with any website you crawl with this tool, or with Apify.

***

If this tool saved you time, we would be grateful for a short review on this page; it helps other people find it. If something did not work as expected, please open an issue on the **Issues** tab and we will look into it quickly.

# Actor input Schema

## `startUrls` (type: `array`):

Pages to convert and, if crawling is on, where the crawl starts. A bare domain such as example.com also works. You can also enter a sitemap URL (…/sitemap.xml), an llms.txt URL, or a direct link to a PDF or DOCX file.

## `crawlLinks` (type: `boolean`):

Also convert pages linked from the start pages (same website only). Turn off to convert only the URLs you listed (plus sitemap / llms.txt URLs if enabled).

## `maxPages` (type: `integer`):

Stop after saving this many pages and documents in total. Your start URLs are always included.

## `maxDepth` (type: `integer`):

How many clicks away from a start URL the crawler may go. 0 = only the start URLs, 1 = pages they link to, and so on. Pages found in sitemaps or llms.txt count as depth 1.

## `stayWithinStartPath` (type: `boolean`):

Only follow links under the start URL's path. For example, starting at example.com/docs/ only crawls pages under /docs/. Also applies to sitemap and llms.txt URLs.

## `includeUrlPatterns` (type: `array`):

Optional. Only crawl discovered URLs that match at least one of these glob patterns, e.g. https://example.com/docs/\*\* or */blog/*. \* matches any characters including /. A pattern without wildcards matches if the text appears anywhere in the URL. Start URLs are always crawled.

## `excludeUrlPatterns` (type: `array`):

Optional. Never crawl URLs matching these patterns, e.g. */tag/*, \*\*/login, ?page=. Also applies to linked files.

## `useSitemaps` (type: `boolean`):

Also read the website's sitemaps (from robots.txt, or /sitemap.xml) and queue the pages they list that are within scope. Finds pages that are not linked from the start page.

## `useLlmsTxt` (type: `boolean`):

Also read the website's /llms.txt file (if it has one) and queue the pages and Markdown files it lists.

## `renderJavaScript` (type: `string`):

Auto downloads pages directly (fast, cheapest) and opens a real browser only for pages that come back empty because they are built with JavaScript; if most pages on a site need the browser, it switches to browser-first for that site. Always uses a browser for every page (slower, charged as browser-rendered pages). Never skips the browser entirely.

## `contentExtraction` (type: `string`):

Main content: keeps the page's main area and removes menus, sidebars, footers, banners and other boilerplate (recommended for docs and most sites). Article text: Readability-style extraction with trafilatura, best for blogs and news; falls back to Main content if it finds too little. Full page: keeps everything except scripts, styles and the elements in 'Remove elements'.

## `removeElementsCssSelector` (type: `string`):

Elements removed before extraction. The default removes navigation, footers, sidebars, dialogs and common cookie-consent banners. Clear it to keep them. Elements holding most of the page's text are never removed.

## `keepElementsCssSelector` (type: `string`):

Optional. If set, only the matching elements are converted (e.g. article, .docs-content, #main). Pages where nothing matches fall back to normal extraction.

## `minWordCount` (type: `integer`):

Pages whose content has fewer words than this are not saved (and not charged). 0 keeps everything.

## `skipDuplicates` (type: `boolean`):

Do not save (or charge for) a page whose text is identical to a page already saved, or whose canonical URL points to a page already saved.

## `saveMarkdown` (type: `boolean`):

Include the content as Markdown (headings, lists, tables, links and code blocks kept).

## `includePlainText` (type: `boolean`):

Include the content as plain text (Markdown formatting removed).

## `saveHtml` (type: `boolean`):

Also include the cleaned main-content HTML.

## `chunkSize` (type: `integer`):

If above 0, each result also gets a 'chunks' array: the Markdown split into pieces of about this many characters on heading, paragraph and sentence boundaries, each with its heading path. 1,000–2,000 characters suits most embedding models. 0 = off.

## `chunkOverlap` (type: `integer`):

How many characters from the end of one chunk are repeated at the start of the next (not across headings). At most half the chunk size.

## `downloadFiles` (type: `boolean`):

Also download PDF and Word (.docx) files linked from crawled pages on the same website (or its subdomains) and convert them to Markdown and text. Each converted file counts as one page. PDF and DOCX start URLs are always converted.

## `maxFileSizeMb` (type: `integer`):

Larger files are skipped.

## `maxFilePages` (type: `integer`):

Only the first this-many pages of each PDF are converted.

## `modifiedSince` (type: `string`):

Optional date (2025-01-31 or 2025-01-31T12:00:00Z). Sitemap entries with an older <lastmod> are not fetched, and pages whose Last-Modified header or modified/published date is older are not saved. Pages with no date are kept. Start URLs are always fetched.

## `incrementalStoreName` (type: `string`):

Optional name of a key-value store in your account (created if missing). The crawler stores a hash of every page's text there; on the next run with the same name, pages whose text has not changed are not saved or charged, and saved pages get changeStatus 'new' or 'changed'.

## `maxConcurrency` (type: `integer`):

How many pages to fetch in parallel. Requests to the same site are also spaced out politely, and any Crawl-delay in robots.txt is honoured.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://docs.python.org/3/tutorial/index.html"
    }
  ],
  "crawlLinks": true,
  "maxPages": 20,
  "maxDepth": 2,
  "stayWithinStartPath": true,
  "includeUrlPatterns": [],
  "excludeUrlPatterns": [],
  "useSitemaps": false,
  "useLlmsTxt": false,
  "renderJavaScript": "auto",
  "contentExtraction": "smart",
  "removeElementsCssSelector": "nav, footer, aside, [role=navigation], [role=contentinfo], [role=dialog], [aria-modal=true], #onetrust-consent-sdk, #CybotCookiebotDialog, #usercentrics-root, .cc-window, .cky-consent-container, .qc-cmp2-container, #cookie-banner, #cookie-notice, #cookie-consent, .cookie-banner, .cookie-notice, .cookie-consent",
  "minWordCount": 0,
  "skipDuplicates": true,
  "saveMarkdown": true,
  "includePlainText": true,
  "saveHtml": false,
  "chunkSize": 0,
  "chunkOverlap": 150,
  "downloadFiles": false,
  "maxFileSizeMb": 20,
  "maxFilePages": 300,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://docs.python.org/3/tutorial/index.html"
        }
    ],
    "removeElementsCssSelector": "nav, footer, aside, [role=navigation], [role=contentinfo], [role=dialog], [aria-modal=true], #onetrust-consent-sdk, #CybotCookiebotDialog, #usercentrics-root, .cc-window, .cky-consent-container, .qc-cmp2-container, #cookie-banner, #cookie-notice, #cookie-consent, .cookie-banner, .cookie-notice, .cookie-consent"
};

// Run the Actor and wait for it to finish
const run = await client.actor("plotpointtech/website-to-markdown-crawler").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://docs.python.org/3/tutorial/index.html" }],
    "removeElementsCssSelector": "nav, footer, aside, [role=navigation], [role=contentinfo], [role=dialog], [aria-modal=true], #onetrust-consent-sdk, #CybotCookiebotDialog, #usercentrics-root, .cc-window, .cky-consent-container, .qc-cmp2-container, #cookie-banner, #cookie-notice, #cookie-consent, .cookie-banner, .cookie-notice, .cookie-consent",
}

# Run the Actor and wait for it to finish
run = client.actor("plotpointtech/website-to-markdown-crawler").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://docs.python.org/3/tutorial/index.html"
    }
  ],
  "removeElementsCssSelector": "nav, footer, aside, [role=navigation], [role=contentinfo], [role=dialog], [aria-modal=true], #onetrust-consent-sdk, #CybotCookiebotDialog, #usercentrics-root, .cc-window, .cky-consent-container, .qc-cmp2-container, #cookie-banner, #cookie-notice, #cookie-consent, .cookie-banner, .cookie-notice, .cookie-consent"
}' |
apify call plotpointtech/website-to-markdown-crawler --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,plotpointtech/website-to-markdown-crawler"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PP4j9pHwcNvRtGwof/builds/XZMaeqkcpK2C5l7DA/openapi.json