Pricing

from $1.00 / 1,000 web page saveds

Try for free

Go to Apify Store

Website to Markdown Crawler for AI & RAG (with PDFs)

Try for free

Crawl a website and save every page as clean Markdown for LLMs, RAG and AI chatbots: main content only, no menus or cookie banners. Linked PDF, Word, PowerPoint and Excel files are converted too. Optional chunking.

Pricing

from $1.00 / 1,000 web page saveds

Rating

0.0

(0)

Developer

Prometheus

Actor stats

Bookmarked

Total users

Monthly active users

12 hours ago

Last modified

Website to Markdown Crawler for LLM & RAG, with PDFs API through CLI

The Apify CLI is the official tool that allows you to use Website to Markdown Crawler for AI & RAG (with PDFs) locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Website to Markdown Crawler for AI & RAG (with PDFs) API in Python

Website to Markdown Crawler for AI & RAG (with PDFs) API in JavaScript

Website to Markdown Crawler for AI & RAG (with PDFs) OpenAPI definition

Website to Markdown Crawler for AI & RAG (with PDFs) API

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

341

5.0

(2)

Website Content Crawler — Text, Markdown & AI Chunks

agency-shift/web-content-crawler

Extract text or clean Markdown from public HTML websites. Create source-linked sections for AI knowledge bases while keeping one row per page. Preserve headings, code and tables; inspect a crawl report for failed URLs and truncated content. No JavaScript rendering.

Mako

Website Content Crawler

worshipful_knife/website-content-crawler

Deep crawl websites and extract clean text, Markdown, or HTML for LLMs, RAG, and AI apps. Removes navigation, ads, cookie banners. Supports headless browser & HTTP. Sitemap discovery, URL scoping, file downloads. Feed ChatGPT, LangChain, LlamaIndex, Pinecone. The cheapest content crawler on Apify.

kata Kuri

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

Website Content Crawler - Markdown, 45 Fields

parseforge/website-content-crawler

Crawls websites from start URLs with depth, domain and URL-pattern limits. 45 fields per page: Markdown and plain text, word count, title and meta tags, headings, Open Graph and JSON-LD, author and dates, emails, phones and social links, detected technologies, links and HTTP status. Ready for LLMs.

ParseForge

187

Website to Markdown Converter

lofomachines/website-to-markdown-converter

Best faster and cheaper way to convert any web page into clean, structured, LLM-ready Markdown.

Lofomachines

Doc To Markdown

abotapi/doc-to-markdown

Convert documents (PDF, Word, PowerPoint, Excel, HTML, images) to clean Markdown. Supports batch processing, metadata extraction, and customizable output formatting.

Abot API

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.