Agent Ready Data Cleaner

Web Scraper RAG Ready

traorealexy/Web-Sraper-RAG-Ready

Turn any website into clean, token-efficient Markdown ready for RAG and LLM pipelines. Removes boilerplate, handles JavaScript rendering, and outputs structured JSON for LangChain, LlamaIndex, and vector databases.

Article Extractor

tugelbay/article-extractor

Extract clean article content from any URL. Get title, author, date, text, images, and metadata as Markdown or plain text. Removes ads and boilerplate. Perfect for LLM training data, RAG pipelines, and AI agents.

RAG Web Extractor — Chunked Content for AI Pipelines

junipr/rag-web-extractor

Extract clean markdown from websites for RAG pipelines. Strip nav, ads, boilerplate. Preserve headings, links, images. Recursive crawling with depth control. Chunked output for embedding pipelines. Build AI knowledge bases.

AI Content Scraper & Cleaner — Scrapes structured content (documentation, articles, FAQs, blog posts) and converts it into clean, normalized JSON datasets for LLM training. Extracts text, detects content types, estimates tokens, and removes boilerplate to produce ready-to-use training data.

Crawl any website, extract clean text, split into chunks with quality scoring, and export to JSON, Pinecone, or Qdrant. Built for RAG pipelines and AI training data. Includes language detection, content type classification, and token counting.

Crawl any website and extract clean, boilerplate-free Markdown optimized for LLMs, RAG pipelines, and AI training datasets. Uses Mozilla Readability to strip navigation and ads, then converts to clean Markdown. No browser required — fast and cheap.

Extract real-time crypto token prices, trading volume, liquidity, and transaction data from DexScreener across 80+ blockchains. 5 modes: search tokens, lookup pairs, get token pairs, browse boosted tokens, and fetch token profiles. Pay-per-event pricing at $2.00 per 1,000 results.

Website Scraper API

kindred_sheng/stealthscrape-api

Give any URL and get back clean Markdown text. Perfect for AI agents, LLM pipelines, and anyone who needs live web data without the HTML clutter.

Instantly extracts clean, ad-free text from any URL. Designed for AI Agents, RAG pipelines, and LLM context windows.

Website Content Crawler

crawlerbros/website-content-crawler

Crawls websites and extracts clean text, markdown, or HTML content. Ideal for LLM training data, RAG pipelines, and knowledge base building.