Pricing

$1.00 / 1,000 page converteds

Try for free

Go to Apify Store

Website to Markdown Crawler for RAG & llms.txt — Docsieve

Try for free

Website to Markdown crawler for RAG and LLMs: crawl any website or docs site and get clean Markdown per page (no menus or footers), RAG-ready chunks with token counts, and llms.txt + llms-full.txt. Monitor mode returns only changed pages. $1 per 1,000 pages.

Pricing

$1.00 / 1,000 page converteds

Rating

0.0

(0)

Developer

CyberMax

Actor stats

Bookmarked

Total users

Monthly active users

4 hours ago

Last modified

Categories

Developer tools

Open source

Security

You can access the Website to Markdown Crawler for RAG & llms.txt — Docsieve programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "startUrls": ["https://docs.apify.com/academy"],
10    "maxPages": 10,
11    "includePaths": ["/academy*"],
12    "proxyConfiguration": { "useApifyProxy": False },
13}
14
15# Run the Actor and wait for it to finish
16run = client.actor("cybermax/website-to-markdown").call(run_input=run_input)
17
18# Fetch and print Actor results from the run's dataset (if there are any)
19print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
20for item in client.dataset(run.default_dataset_id).iterate_items():
21    print(item)
22
23# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Website to Markdown Crawler for RAG, LLMs & llms.txt API in Python

The Apify API client for Python is the official library that allows you to use Website to Markdown Crawler for RAG & llms.txt — Docsieve API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Website to Markdown Crawler for RAG & llms.txt — Docsieve API in JavaScript

Website to Markdown Crawler for RAG & llms.txt — Docsieve API through CLI

Website to Markdown Crawler for RAG & llms.txt — Docsieve OpenAPI definition

Website to Markdown Crawler for RAG & llms.txt — Docsieve API

🔥 FireScrape AI Website Content Markdown Scraper

mohamedgb00714/fireScraper-AI-Website-Content-Markdown-Scraper

Advanced web scraper powered by Crawlee and Puppeteer — extracts website content, converts it to Markdown, and structures it for LLM training datasets.

mohamed el hadi msaid

305

1.9

(2)

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

341

5.0

(2)

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Logiover

Website Content Crawler to Markdown for RAG & LLMs

rod_analytics/website-to-markdown

Low-cost website content crawler: turn any site into clean Markdown for RAG, LLMs and AI agents at a flat $0.30 per 1,000 pages. Main-content extraction, GFM tables, RAG chunks, llms.txt per domain, robots.txt respected.

Rod Services

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Locomint

Website to Markdown — LLM-Ready Content Crawler for RAG

actor-forge/website-to-markdown

Crawl any website and get clean, LLM-ready Markdown for every page: main content only (no menus, footers or cookie banners), absolute links, tables, title, description, author and publish date. For RAG, vector databases, AI agents and training data. Flat $1 per 1,000 pages.

ActorForge

Website & URL to Markdown Crawler + llms.txt Generator

api-co/website-to-markdown

Crawl websites or URL lists into clean Markdown for LLMs, RAG and AI agents: URL to Markdown, docs to Markdown, HTML to Markdown with main-content extraction, metadata and chunks. Includes an llms.txt generator for any site.

Emilio Cisneros

Website to RAG Markdown Crawler

knotted_tussock/rag-markdown-crawler

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Ralph T

Website to Markdown Crawler

vizellie/website-to-markdown-crawler

Crawls a public website and saves clean Markdown for AI tools, docs and content audits. Same-site pages only, with robots.txt checked on every URL, heading-aware chunks, repeated boilerplate removed, and an optional change-only recrawl.

Vi Zellie