Website Content Crawler Pro — Markdown for AI & RAG

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Scrape Instagram profile data. Get followers, following, bio, posts, engagement metrics. No login required. Works with Apify residential proxies. $0.0015/profile — 50% cheaper than the leading alternative.

Scrape Trustpilot reviews for any business. Ratings, text, sentiment, company replies. $0.001/review.

Scrape Google Maps business data. Get names, addresses, phone numbers, websites, ratings, reviews, coordinates. Uses the official Places API ($200/month free from Google). Perfect for lead generation. $0.002/result.

Scrape TikTok videos, profiles, and hashtags. Get likes, views, comments, shares, music, descriptions. Works with Apify residential proxies. $0.001/video — 3x cheaper than alternatives.

Website Crawler API — Markdown for RAG

tugelbay/website-content-crawler

Website crawler API for public pages and clean Markdown, text, or HTML output for RAG pipelines, AI agents, documentation indexing, and monitoring. Guide: https://konabayev.com/tools/website-content-crawler/?utm_source=apify_info&utm_medium=referral&utm_campaign=website-content-crawler

RAG Web Browser

parseforge/rag-web-browser

Give your AI agents real-time web access! Search the web on any topic and get full page content as clean Markdown, ready for LLMs, RAG pipelines, or OpenAI Assistants. Includes titles, descriptions, links, authors, images, and metadata. Start grounding your AI with fresh data in minutes!

Ai Contextual Markdown Transformer

audit-data-solutions/ai-contextual-markdown-transformer

Convert messy HTML, broken data, and unformatted text into clean Markdown. Powered by Llama 3.3 via Groq. Expertly removes web-clutter and restores structure. Perfect for RAG, LLM training, and data cleaning. Ultra-fast, free, and accurate.

Enterprise-grade extraction. Converts complex, Javascript-heavy websites into clean, semantic Markdown optimized for LLMs and RAG pipelines.