Website Crawler - Site Pages to Markdown and Text

Website Markdown Crawler - Content & Change Tracking

benthepythondev/website-markdown-crawler

Crawl public HTML within a website section. Export Markdown, text, source URLs and change hashes, with page limits and a URL-level coverage report.

Website to Markdown Crawler - Full-Site Text for LLMs & RAG

entranced_gelato/website-to-markdown-crawler

Crawl any website from a start URL and get every page as clean text + Markdown for LLMs, RAG, and AI agents. Follows internal links with depth and page limits, strips nav and ads, and returns one structured record per page. A fast, no-config site-to-Markdown crawler.

Website Content Crawler — Clean Markdown for AI

dallel-data/website-content-to-markdown

Crawl public HTML websites into clean Markdown and text for AI, RAG and content research. Export titles, metadata, canonical URLs and links with depth and page limits.

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Crawl any website and extract clean article text and Markdown, ready to feed into LLMs, ChatGPT, vector databases and RAG pipelines. Removes navigation, ads and boilerplate. Configurable crawl depth and page limits. Export to JSON, CSV, Excel.

Crawl any website and extract clean text, Markdown, and HTML from every page — ready for LLM, RAG, and AI ingestion.

Website to Markdown Scraper

receptional_blender/website-to-markdown-scraper

Crawl any website and turn its pages into clean Markdown — plus optional plain text, raw HTML and full-page screenshots. Built for LLM, RAG and AI training datasets.

Fetch any URL and return clean markdown/text plus links. No headless browser for most pages. Free demo: https://vaytric.com/page-to-markdown/ — bulk URL lists here.

Website Crawler Scraper

automation-lab/website-content-crawler

Crawl public websites into page-level metadata, links, clean text, and Markdown for search indexes and LLM knowledge bases.

Website to Markdown Crawler for LLM & RAG (fast, no browser)

webdatatools/website-to-markdown

Website to Markdown Crawler turns any site into clean Markdown per page for LLMs, RAG pipelines and vector DBs — no headless browser, $1 per 1,000 pages.