Tech Docs to LLM-Ready Markdown

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

Newsletter Scraper

benthepythondev/newsletter-scraper

Extract newsletter archives from Substack, Beehiiv, and Ghost platforms. Get full content in markdown format, complete metadata, embedded images, word counts, and AI-ready token counts. Perfect for content research, competitive analysis, and training AI models.

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

Universal Markdown Scraper for LLMs

botflowtech/universal-markdown-scraper-for-llms

Universal Markdown Scraper for LLMs

AI Markdown Maker

onescales/bulk-ai-markdown-maker

Convert any web page into clean, AI ready markdown format in seconds. Perfect for feeding content to AI models, creating documentation, or archiving web content in a portable format. In addition it intelligently parse web content, removing ads, navigation, and other clutter. Generate Markdown Today!

Reddit Scraper

benthepythondev/reddit-scraper

Extract Reddit posts, comments & user data in AI-ready markdown format. No API keys needed! 25% cheaper than competitors. Perfect for AI training, sentiment analysis & market research. Includes bulk comment scraping with progress tracking.

Markitdown Mcp Server

rector_labs/markitdown-mcp-server

Cloud-hosted MCP server converting 29+ document formats (PDF, DOCX, PPTX, images, audio) to AI-ready Markdown. Zero Python setup. Perfect for RAG pipelines and AI agents. Pay-per-use: $0.02/conversion. Built on Microsoft's Markitdown (82k+ ⭐).

/llms.txt Generator

jakub.kopecky/llmstxt-generator

The /llms.txt Generator 🕸️📄 extracts website content to create an llms.txt file for AI apps 🤖✨ like LLM fine-tuning and indexing. Output is available 📥 in the Key-Value Store for easy download and integration into workflows. 🚀

RAG Knowledge Loader

botflowtech/rag-knowledge-loader

RAG Knowledge Loader

Web Scraper 🚀

datascoutapi/web-scraper

Web Scraper Pro extracts clean structured data for LLMs/RAG. Browser-based, 10x faster with anti-detection bypassing Cloudflare/CAPTCHA & proxy rotation. Bulk/recursive crawl 50k URLs at 500 pages/min. JSON/CSV/API, free tier.