Pricing

from $2.00 / 1,000 pages

Website to Markdown — RAG Content & Change Feed

Turn public static HTML into clean Markdown with source URLs, content hashes and word counts for RAG pipelines. Optional same-site crawling and stored-state comparison skip unchanged pages. No LLM API key.

Pricing

from $2.00 / 1,000 pages

Rating

0.0

(0)

Developer

ABDULWAHAB NASER RASHED ALQARAWI

Actor stats

Bookmarked

Total users

Monthly active users

19 hours ago

Last modified

Website Content Crawler — Markdown for LLM & RAG

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Yuliia Kulakova

Web Scraper Mcp

loom-stack/web-scraper-mcp

Batch scrape any website via Model Context Protocol. Works with Claude Desktop, Cursor, Cline, and all MCP-compatible AI clients. Smart retries, JS rendering, clean Markdown output, and async crawling for large batches.

The Loom Stack

AI Web to Markdown - LLM-Ready Extractor

wiry_kingdom/ai-web-to-markdown

Convert any URL into clean LLM-ready markdown. Strips ads, nav, footer. Preserves headings, lists, tables, code blocks. Returns token count. Perfect for RAG, fine-tuning, AI agents. 10x cheaper than Firecrawl.

Mohieldin Mohamed

Knowledge Intelligence Engine — Website to Markdown for RAG

ryanclinton/website-content-to-markdown

Turn any website, documentation site or help centre into a retrieval-ready knowledge corpus for RAG and AI search. Clean Markdown plus chunks, change detection, deduplication, retrieval scoring, version awareness and a full corpus audit, in one run.

Ryan Clinton

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

RSS Feed Scraper & Keyword Monitor

sturdydata/rss-feed-monitor

Turn any list of RSS/Atom feeds — or just website homepages, feeds are auto-discovered — into a clean, deduplicated stream of news items, optionally filtered by keywords. Normalized output across RSS 2.0 and Atom: title, link, published date, author, categories, summary and full content as plain...

Sturdy Data

Website Crawler Scraper

automation-lab/website-content-crawler

Crawl public websites into page-level metadata, links, clean text, and Markdown for search indexes and LLM knowledge bases.

Stas Persiianenko

RAG Freshness - Incremental Vector Updates

brocoswork/omniapi-rag-freshness

Detect web content changes and return only vector upserts and deletes, avoiding unnecessary RAG re-embeddings.

Anxo Brocos

RAG Content Crawler — Any Site to Clean Markdown

eszetael_lab/rag-content-crawler

Turn any website into clean Markdown for your LLM or RAG pipeline. Navigation, cookie banners and boilerplate removed; every page says whether the text was truncated. Pay per delivered page — pages that returned nothing are free.