Docs Change Feed for RAG

A powerful web crawler that extracts text content from websites, optimized for AI models, Large Language Models (LLMs), vector databases, and Retrieval-Augmented Generation (RAG) pipelines.

Llm Ready Documentation Scraper

direct_duty/llm-ready-documentation-scraper

Developers and AI agents need to read documentation (e.g. Stripe Docs, Next.js Docs), but standard scrapers return noisy HTML that includes: navigation bars headers / footers ads / cookie banners This Actor must return pure content-only Markdown, suitable for vectorization and semantic search.

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.

Markdown Maker: HTML to Markdown 📝

shahidirfan/Markdown-Maker

Instantly convert complex HTML into clean, structured Markdown. This lightweight actor is optimized to render web content into a format that is easily readable for AI LLMs, reducing token usage and improving context. Perfect for RAG pipelines and preparing data for training.

Extract text from scanned PDFs and images (PNG, JPEG, TIFF, WebP, GIF, BMP) with Tesseract 5 OCR in 29 languages. PDF pages that already have text are read directly at a tenth of the price. Per-page text and confidence, optional word bounding boxes. $3 per 1,000 OCR pages.

Fetch any URL and return clean markdown/text plus links. No headless browser for most pages. Free demo: https://vaytric.com/page-to-markdown/ — bulk URL lists here.

Crawl any website and get each page's main content as clean Markdown and text for LLMs, RAG and search. Sitemap-seeded, de-duplicated, no nav or cookie banners. Respects robots.txt and AI-crawler opt-outs. $1.50 per 1,000 pages.

Crawls a website and converts every page into clean, LLM-ready Markdown for RAG pipelines, vector databases, and AI agents. Removes nav, ads, and boilerplate. Predictable pricing: $0.004 per page converted.

PDF & DOCX to Markdown Converter for AI Agents

owlabee/pdf-to-markdown

Convert PDF, DOCX, HTML and text documents from URLs into clean Markdown with headings, lists, tables, links and metadata, plus optional heading-aware RAG chunks and OCR for scanned pages. No browser, no external API. Pay per page converted ($0.001).

Article Extractor API — Title, Author, Date & Full Text

highbrow_fame/article-extractor-api

Extracts the article from any news or blog URL: title, author, published date, full text, Markdown, lead image and language. Works on paywall-free pages. Unofficial Article Extractor API.