You can access the Website to Markdown — Content Crawler for LLM & RAG programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "startUrls": [
10        { "url": "https://docs.apify.com/platform" },
11        { "url": "https://crawlee.dev/docs" },
12    ],
13    "maxPages": 50,
14    "maxDepth": 3,
15    "includePathPrefixes": [],
16    "excludePathPatterns": [
17        "\\.(png|jpe?g|gif|svg|webp|pdf|zip|mp4|css|js)$",
18        "/login",
19        "/signup",
20        "\\?replytocom=",
21    ],
22    "outputFormat": "markdown",
23    "removeSelectors": [],
24    "maxConcurrency": 10,
25}
26
27# Run the Actor and wait for it to finish
28run = client.actor("webdatatools/website-to-markdown").call(run_input=run_input)
29
30# Fetch and print Actor results from the run's dataset (if there are any)
31print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
32for item in client.dataset(run.default_dataset_id).iterate_items():
33    print(item)
34
35# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Website to Markdown - Content Crawler for LLM & RAG API in Python

The Apify API client for Python is the official library that allows you to use Website to Markdown — Content Crawler for LLM & RAG API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Website to Markdown — Content Crawler for LLM & RAG API in JavaScript

Website to Markdown — Content Crawler for LLM & RAG API through CLI

Website to Markdown — Content Crawler for LLM & RAG OpenAPI definition

Website to Markdown — Content Crawler for LLM & RAG API

Website Content Extractor - Markdown for LLM, RAG, Vector DB

snow_leo_data/website-to-markdown

Readable text from 73.1% of 52 live pages over plain HTTP, including 10 of 10 documentation sites. Website to LLM text and a website to markdown converter in one run: clean Markdown, chunks with overlap for a vector database, 46 flat fields, and what changed since the last run.

Snow Leo Data

AI Web Search Scraper - SERP & Page Content for LLM RAG

get_anything/ai-web-search-scraper

Web search API for LLMs: query -> clean results (title, URL, snippet, position, domain) and OPTIONALLY each page's full content as LLM-ready Markdown. Ground ChatGPT/Claude and RAG pipelines with live web results. Google (browser-rendered) or DuckDuckGo engine. Export JSON, CSV or Markdown.

Get Anything

Google Search Results (SERP) & Ads Scraper

scrapeforgehq/google-serp-scraper

Scrape Google SERPs at scale: paid ads (headline, copy, displayed/final URL), organic results, People Also Ask, and related searches — by keyword, country, language, and device. Clean JSON output for SEO, PPC, and competitor ad research.

ScrapeForge

Markdown to HTML Converter — Bulk

hipersoft/markdown-to-html

Convert Markdown to clean, styled HTML with GitHub-flavored tables, code highlighting and an optional full HTML document wrapper. For emails, CMS content and static sites.

hiper soft

Website to Markdown — Clean Page Text for LLMs

sourcedirect/website-to-markdown

Fetch any URL and return clean markdown/text plus links. No headless browser for most pages. Free demo: https://vaytric.com/page-to-markdown/ — bulk URL lists here.

Kurt Landman

Web Page to Markdown for AI: $1/1K Pages, No Login

conserving_celerytop/web-page-to-markdown

Turn any list of web pages into clean Markdown for LLMs, RAG and AI agents: main content without menus and ads, tables kept, plus title, description, language, dates and links. Plain HTTP first, JavaScript only when a page needs it. $1 per 1,000 pages, failures free.

Don Mangu

Website Content to Markdown & llms.txt Crawler - AI & RAG Ready

abi4/website-content-llmstxt-crawler

High-performance, lightweight asynchronous crawler converting web pages into clean Markdown, computing token estimates, and generating standard /llms.txt manifests for AI and RAG ingestion.

Abi

Web Crawler: Website to Markdown for LLM & RAG

scrape-lads/website-content-crawler-rag

Website content crawler for AI. Crawl any site into clean Markdown, text and ready-to-embed RAG chunks for LLMs and vector databases. Removes navigation, footers and cookie banners, renders JavaScript only when needed. $2 per 1,000 pages.

Scrape Lads

URL to Agent Markdown

encouraged_offspring/url-to-agent-markdown

Lav Chaudhary

Website to Markdown Crawler for AI, RAG & LLMs

dima_kadirovich/website-to-markdown

Crawl any website and get clean main-content Markdown for every page: no menus or footers. RAG-ready chunks, llms.txt, and an 'only changed pages' mode for cheap scheduled refreshes. $1 per 1,000 pages, all included.