LLM Website Crawler: Clean Markdown & RAG Chunks

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

Crawl any website and extract clean text, Markdown, and HTML from every page — ready for LLM, RAG, and AI ingestion.

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Crawl any website and extract clean article text and Markdown, ready to feed into LLMs, ChatGPT, vector databases and RAG pipelines. Removes navigation, ads and boilerplate. Configurable crawl depth and page limits. Export to JSON, CSV, Excel.

TikTok Video Script Analyzer: Hook, On-Screen Text & CTA

zaiq/tiktok-video-script-analyzer

Any TikTok video as its script: the words said (timed), the text on screen, the hook and its type, the beats, the call to action, format, topics, summary and why it works, plus plays, likes and shares. Video links, or a creator's top outliers. No login.

Crawls Spanish-language websites and extracts clean markdown content optimized for LLM RAG pipelines. Handles Spanish character encoding, removes boilerplate, and outputs structured chunks ready for embedding.

Website Content Crawler to Markdown by Country

zaiq/website-crawler-by-country

Crawl a website and get each page's main content as clean Markdown or text for RAG, LLM and AI agents, fetched from the country you pick through residential (home) connections. Geo-targeted, with proof of the country for every page. JavaScript when needed. Blocked pages are free.

Fetch any URL and return clean markdown/text plus links. No headless browser for most pages. Free demo: https://vaytric.com/page-to-markdown/ — bulk URL lists here.

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.