Website RAG & Lead Intelligence Crawler

Website to Markdown Crawler – Clean RAG Content

signal_lab/website-to-markdown-crawler

Website-to-Markdown crawler for RAG and LLM datasets. Extract clean content, SEO metadata, links, JSON-LD, sitemaps, and robots.txt-compliant pages.

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

Website Contact Crawler

competent_clarinet/website-contact-crawler

Crawls websites to extract emails, phones, and social links.

Crawl public websites for emails, phones, social profiles, contact pages, and company metadata.

Crawl a whole website or docs site and get one clean, LLM-ready Markdown + JSON record per page (title, headings, content, links, token count). Built for RAG ingestion and AI knowledge bases.

Website to RAG Dataset

sebastian-actors/website-to-rag-dataset

Convert public websites, docs, blogs, and XML sitemaps into clean Markdown, structured metadata, and stable chunks for RAG pipelines and vector databases.

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.