Pricing

from $1.00 / 1,000 page crawleds

Website to Markdown Crawler | Pre-Chunked for RAG

Crawl any website into clean, pre-chunked Markdown with per-chunk token counts for RAG pipelines, vector DBs (Pinecone, Qdrant) and LLM context. MCP native for Claude & ChatGPT. SPA support via Playwright. Pay only for pages that crawl. A Firecrawl alternative.

Pricing

from $1.00 / 1,000 page crawleds

Rating

0.0

(0)

Developer

The Mine Works

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

RAG Website Crawler - Markdown Chunks for LLMs & MCP API through CLI

The Apify CLI is the official tool that allows you to use Website to Markdown Crawler | Pre-Chunked for RAG locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Website to Markdown Crawler | Pre-Chunked for RAG API in Python

Website to Markdown Crawler | Pre-Chunked for RAG API in JavaScript

Website to Markdown Crawler | Pre-Chunked for RAG OpenAPI definition

Website to Markdown Crawler | Pre-Chunked for RAG API

Website to RAG Chunks

cynix_dev/web-to-rag-chunks

Crawl any website and turn its pages into clean, chunked, metadata-rich Markdown records ready for RAG pipelines, vector stores, and custom GPTs.

Cynix Dev

RAG-Ready Website Crawler — Clean Content for LLMs & Vector DBs

yourwingman/rag-ready-crawler

Crawl websites and output clean, chunked content optimized for RAG pipelines, LLM training data, and vector databases. Built for AI knowledge bases and semantic search.

Wingman

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Logiover

Website Content Crawler - Markdown & Text for LLM / RAG

pear_fight/website-content-crawler-markdown-text-for-llm-rag

Crawl any website and extract clean article text and Markdown, ready to feed into LLMs, ChatGPT, vector databases and RAG pipelines. Removes navigation, ads and boilerplate. Configurable crawl depth and page limits. Export to JSON, CSV, Excel.

Harald

Website to Markdown for LLMs

agentictools/website-to-markdown-llm

Crawl a site and export clean Markdown with token counts and chunks, ready for RAG.

Ken Agland

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Hitman studio

Website to RAG Markdown Crawler

knotted_tussock/rag-markdown-crawler

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Ralph T

AI Data Pipeline — Crawl, Chunk & Export to Vector DB

ozapp/ai-data-pipeline

Crawl any website, extract clean text, split into chunks with quality scoring, and export to JSON, Pinecone, or Qdrant. Built for RAG pipelines and AI training data. Includes language detection, content type classification, and token counting.

Ozapp

Website Content Crawler for AI & LLM Data

your_scraper_guy/website-content-crawler-lite

Crawl any website from a seed URL and extract clean Markdown content, ready for LLM training data, RAG pipelines, and vector databases. Set crawl depth, page limits, and domain scope.