Website to Markdown Crawler for RAG & llms.txt — Docsieve

🔥 FireScrape AI Website Content Markdown Scraper

mohamedgb00714/fireScraper-AI-Website-Content-Markdown-Scraper

Advanced web scraper powered by Crawlee and Puppeteer — extracts website content, converts it to Markdown, and structures it for LLM training datasets.

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Website Content Crawler to Markdown for RAG & LLMs

rod_analytics/website-to-markdown

Low-cost website content crawler: turn any site into clean Markdown for RAG, LLMs and AI agents at a flat $0.30 per 1,000 pages. Main-content extraction, GFM tables, RAG chunks, llms.txt per domain, robots.txt respected.

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Crawl any website and get clean, LLM-ready Markdown for every page: main content only (no menus, footers or cookie banners), absolute links, tables, title, description, author and publish date. For RAG, vector databases, AI agents and training data. Flat $1 per 1,000 pages.

Website & URL to Markdown Crawler + llms.txt Generator

api-co/website-to-markdown

Crawl websites or URL lists into clean Markdown for LLMs, RAG and AI agents: URL to Markdown, docs to Markdown, HTML to Markdown with main-content extraction, metadata and chunks. Includes an llms.txt generator for any site.

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Crawls a public website and saves clean Markdown for AI tools, docs and content audits. Same-site pages only, with robots.txt checked on every URL, heading-aware chunks, repeated boilerplate removed, and an optional change-only recrawl.

Turn any website into clean Markdown for RAG and agents. Scrape a page or crawl a site. robots.txt honored. Brainiall Web engine.