Website to Markdown Crawler – Clean RAG Content

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Website to Markdown for LLMs & RAG — Content Extractor

fetchbase/website-to-markdown

Turn any URL or whole site into clean, LLM-ready Markdown, text, or JSON. Strips nav/ads/boilerplate; keeps headings, links, tables, code. Sitemap-aware concurrent crawl, URL filters, robots.txt respected, rich metadata. Charged only per page extracted — no startup fee.

Convert any public web page to clean, LLM-ready Markdown with metadata — by URL, a list of URLs, or a whole-site crawl. Strips nav/ads/boilerplate, keeps headings/lists/tables/code. Respects robots.txt. No API key.

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Website to Markdown Scraper — LLM & RAG Ready

perforated_hummingbird/url-to-markdown

Scrape any website into clean, LLM-ready Markdown. This URL-to-Markdown converter strips ads, nav, and boilerplate with Mozilla Readability — feed your AI models and RAG pipelines only real content. Batch URLs, optional JavaScript rendering, pay only for pages scraped successfully.

AST Website Content Crawler for RAG

lukas459/ai-web-to-markdown-crawler-llm-rag-optimized

Advanced web crawler extracting token-chunked Markdown for LLMs & RAG. Features dual-engine hybrid parsing (IBM Docling & Trafilatura) to preserve tables and layouts. Delivers clean text split by native tiktoken counts with rich metadata (headings, levels, tokens) for instant vector search.

Convert web pages and HTML to clean Markdown for RAG, LLM training, AI knowledge bases, and content migration. Strips nav, ads, scripts, and styling while preserving structure. Bulk output includes word/link/image counts.

Smart Article Extractor - Clean Text & Markdown from Any URL

ninhothedev/smart-article-extractor

$1/1K 🔥 Smart article extractor! Clean title, author, date, text & Markdown from any article URL — perfect for RAG & LLMs. No key. JSON, CSV, Excel or API in seconds. Feed news & blogs into AI in seconds ⚡

Paste a homepage or sitemap and get clean Markdown, metadata, JSONL chunks, and source URLs for RAG at a low per-page price.

Crawl any website and get clean markdown, text, or HTML per page, ready for RAG pipelines, chatbots, and LLM fine tuning. Plain HTTP, no browser, no API key. Pay per page.