AI Training Data Collector — Clean Web Datasets for LLMs

RAG Pipeline Data Collector

scraper_guru/rag-pipeline-data-collector

AI-ready web content extraction for RAG systems, LLMs, and AI agents. Single-page or multi-page scraping with parallel processing.

Website To Markdown

smart_api/website-to-markdown

Convert any webpage into clean, LLM-ready Markdown in seconds — perfect for AI training data, RAG pipelines, and content archiving.

The ultimate web-to-markdown tool for AI builders. Extracts clean content from any site, auto-dismisses cookie banners, and handles SPAs with Playwright. Optimized for LangChain, LlamaIndex, and RAG pipelines. Save token costs with 99% noise-free markdown.

Ultra Fast Shopify Scraper

rover-omniscraper/shopify-scraper

Universal Shopify Scraper for Apify — Extract product data from any Shopify store via API. Multi-store support, auto-detection, smart filtering by price/tags/sales, new arrival flags, sale detection. Handles pagination, configurable request delays, pure HTTP (no browser). Perfect for e-com research

Firecrawl Search - LLM-ready content

alizarin_refrigerator-owner/firecrawl-search---llm-ready-content

Search the web and get clean, LLM-ready content in one API call. Powered by Firecrawl's /v1/search endpoint. Returns markdown, HTML, or extracted data. Perfect for SEO research, competitor analysis, and AI training data collection

Facebook Group Scraper - Posts, Comments & Reactions

whoareyouanas/facebook-group-scraper

Scrape posts from public & private Facebook groups. Extract post text, author info, reactions, comments, shares, images, videos, and group metadata. Cookie auth for private groups. Clean JSON/CSV output. No API key needed.

Reddit Scraper - Posts, Comments & Users

betterdevsscrape/reddit-scraper

Extract posts, comments, communities & user profiles from any subreddit at scale. Fetches all comments including hidden/collapsed ones. Breaks Reddit's 1000-post limit with date windowing. No login needed, no browser. $0.003 per result. Supports search, sorting, NSFW filtering & date filtering.

Quick Website Content Scraper ( Extract Text for RAG & LLMs )

automateitplease/ai-web-content-scraper-extract-text-for-rag-llms

Extract clean text from any website for AI/LLM applications. Supports both static and JavaScript-rendered sites (React, Vue, Angular). Perfect for RAG systems, chatbot training, and content analysis.

Web Scraper 🚀

datascoutapi/web-scraper

Web Scraper Pro extracts clean structured data for LLMs/RAG. Browser-based, 10x faster with anti-detection bypassing Cloudflare/CAPTCHA & proxy rotation. Bulk/recursive crawl 50k URLs at 500 pages/min. JSON/CSV/API, free tier.

AI Training Data Curator

ryanclinton/ai-training-data-curator

Crawl any website and extract clean, structured text data ready for LLM fine-tuning, RAG pipelines, and AI model training.