Convert a documentation site to Markdown chunks for RAG

Created by

Teddy Wasserman

Actor

Website to Markdown - Content Extractor for LLM & RAG

Crawls a documentation site from one start URL, extracts the main content of every page as clean Markdown (no menus or footers) and splits it into 1,500-character chunks with overlap, ready for a vector database or a chatbot knowledge base.

Try for free

Website to Markdown - Content Extractor for LLM & RAGteddythebuilder/website-to-markdown

URL

Title

Words

Language

+4 fields

Input

Start URLs(required)

url:https://docs.python.org/3/tutorial/index.html

Crawl depth:1

Max pages:20

RAG chunk size (characters):1500

Chunk overlap (characters):150

Output fields

URL

Title

Words

Language

Published

Markdown

Error (not charged)

Warning

How it works

Sign up on Apify or log in

The Website to Markdown - Content Extractor for LLM & RAG Actor is ready to run with the inputs already set for "Convert a documentation site to Markdown chunks for RAG". Sign up or log in and you'll land in Apify Console with a copy of this exact task.

Start the "Convert a documentation site to Markdown chunks for RAG" task

The Website to Markdown - Content Extractor for LLM & RAG Actor will start running based on the input automatically. You can also change all the input fields, like start urls, crawl depth, and more, to match your specific use case.

Receive the "Convert a documentation site to Markdown chunks for RAG" output

Monitor the progress in real time. You will be notified as soon as your dataset is complete and ready for review. Each result includes data such as URL, title, words, and more.