Mintlify Docs Scraper — Docs to Clean Markdown

llms-txt-converter is an Apify Actor that crawls any documentation site and outputs the two GEO-standard files (llms.txt and llms-full.txt) used by ChatGPT, Claude, Perplexity, Gemini, and Bing Copilot to find and cite your docs.

Website to Markdown for LLMs and RAG

rodrgds/website-to-markdown

Convert webpages into clean markdown for LLMs, RAG pipelines, AI datasets, archives, and content extraction. Simple pay-per-page pricing.

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Crawl a site and export clean Markdown with token counts and chunks, ready for RAG.

Website to Markdown Crawler - Full-Site Text for LLMs & RAG

entranced_gelato/website-to-markdown-crawler

Crawl any website from a start URL and get every page as clean text + Markdown for LLMs, RAG, and AI agents. Follows internal links with depth and page limits, strips nav and ads, and returns one structured record per page. A fast, no-config site-to-Markdown crawler.

Universal Markdown Scraper for LLMs

botflowtech/universal-markdown-scraper-for-llms

Universal Markdown Scraper for LLMs

Find public llms.txt, llms-full.txt, and AI-readable documentation links for company domains.

LLMs.txt & llms-full.txt Generator for Any Website

fayoussef/llms-txt-generator

Generate a spec-compliant llms.txt and llms-full.txt for any website in one click. Crawls your sitemap or site, extracts titles, descriptions and full Markdown content, and outputs ready-to-upload files that make your site visible to ChatGPT, Claude, Perplexity and other AI search engines.

Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output. For AI readiness, SEO for AI, documentation portals, and archiving.

Webpage to Markdown

epicscrapers/webpage-to-markdown

Get the main content of any page as Markdown. Great for LLMs and AI agent workflows.