Website to Markdown — Content Crawler for LLM & RAG

Website Content Extractor - Markdown for LLM, RAG, Vector DB

snow_leo_data/website-to-markdown

Readable text from 73.1% of 52 live pages over plain HTTP, including 10 of 10 documentation sites. Website to LLM text and a website to markdown converter in one run: clean Markdown, chunks with overlap for a vector database, 46 flat fields, and what changed since the last run.

AI Web Search Scraper - SERP & Page Content for LLM RAG

get_anything/ai-web-search-scraper

Web search API for LLMs: query -> clean results (title, URL, snippet, position, domain) and OPTIONALLY each page's full content as LLM-ready Markdown. Ground ChatGPT/Claude and RAG pipelines with live web results. Google (browser-rendered) or DuckDuckGo engine. Export JSON, CSV or Markdown.

Google Search Results (SERP) & Ads Scraper

scrapeforgehq/google-serp-scraper

Scrape Google SERPs at scale: paid ads (headline, copy, displayed/final URL), organic results, People Also Ask, and related searches — by keyword, country, language, and device. Clean JSON output for SEO, PPC, and competitor ad research.

Markdown to HTML Converter — Bulk

hipersoft/markdown-to-html

Convert Markdown to clean, styled HTML with GitHub-flavored tables, code highlighting and an optional full HTML document wrapper. For emails, CMS content and static sites.

Fetch any URL and return clean markdown/text plus links. No headless browser for most pages. Free demo: https://vaytric.com/page-to-markdown/ — bulk URL lists here.

Web Page to Markdown for AI: $1/1K Pages, No Login

conserving_celerytop/web-page-to-markdown

Turn any list of web pages into clean Markdown for LLMs, RAG and AI agents: main content without menus and ads, tables kept, plus title, description, language, dates and links. Plain HTTP first, JavaScript only when a page needs it. $1 per 1,000 pages, failures free.

High-performance, lightweight asynchronous crawler converting web pages into clean Markdown, computing token estimates, and generating standard /llms.txt manifests for AI and RAG ingestion.

Web Crawler: Website to Markdown for LLM & RAG

scrape-lads/website-content-crawler-rag

Website content crawler for AI. Crawl any site into clean Markdown, text and ready-to-embed RAG chunks for LLMs and vector databases. Removes navigation, footers and cookie banners, renders JavaScript only when needed. $2 per 1,000 pages.

Website to Markdown Crawler for AI, RAG & LLMs

dima_kadirovich/website-to-markdown

Crawl any website and get clean main-content Markdown for every page: no menus or footers. RAG-ready chunks, llms.txt, and an 'only changed pages' mode for cheap scheduled refreshes. $1 per 1,000 pages, all included.