Website Content Crawler - Markdown, 45 Fields

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Scrape Failed Killer! A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

Etsy Review Scraper

getdataforme/etsy-review-scraper

The Etsy Review Scraper is an easy scraper to extract reviews from specified Etsy shop. The Etsy review scraper is easy to use, simply provide the shop name and the scraper will do the magic providing you the reviews from the products. etsy scraper python

Extract text, metadata, and page-by-page content from PDF files. Provide PDF URLs and get structured JSON with full text, per-page text, page count, author, title, creation date, and more. Export as JSON, CSV, or Excel. No browser or proxy needed.

RAG Web Browser

apify/rag-web-browser

Web search and fetch tool for AI agents and RAG pipelines. It queries Google Search, scrapes the top N pages using a full web browser, and returns their content as clean Markdown for further processing by an LLM. Can also fetch individual URLs.

RAG Web Browser Scraper

datapilot/rag-web-browser-scraper

RAG Web Browser Search & Crawl Actor uses to search Bing or crawl URLs, then extracts page content as clean markdown. It captures title, description, language, HTTP status, and structured metadata. Supports multiple queries, proxies, and outputs organized crawl + search results.

SuperScraper API

apify/super-scraper-api

Generic REST API for scraping websites: send a URL and get back HTML. This Actor is a drop-in replacement for ScrapingBee, ScrapingAnt, and ScraperAPI services. And it is open-source!

🛍️ Etsy Product & Reviews Scraper

dz_omar/etsy-product-reviews-scraper

🆓 Free for now —🛍️ Scrape full Etsy product & listing data from any URL pricing, shop, variations, shipping, policies, ratings & media plus optional ⭐ buyer reviews with rating, media & sort filters. Public data, no login.

PDF Scraper

onidivo/pdf-scraper

Scrape and extract text from PDF links.