Pricing

from $0.57 / 1,000 pages

Try for free

Go to Apify Store

Website Intelligence Crawler

Try for free

Crawl a public website and get its pages as clean text and Markdown, plus chunks sized for embedding with token estimates, title, description, language and same-site links. Follows robots.txt. No model, no browser. $0.60 per 1,000 pages.

Pricing

from $0.57 / 1,000 pages

Rating

0.0

(0)

Developer

Dami's Studio

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Free

up to

8,333

pages

/ month

$5 to spend in Apify Store or on your own Actors

Starter

$0.59

/ 1,000 pages

$19

/ month + pay as you go

$19 to spend in Apify Store or on your own Actors

Scale

$0.58

/ 1,000 pages

$199

/ month + pay as you go

$199 to spend in Apify Store or on your own Actors

Business

$0.57

/ 1,000 pages

$999

/ month + pay as you go

$999 to spend in Apify Store or on your own Actors

All events

	Free	Starter	Scale	Business
Page Primary	$0.60 / 1,000	$0.59 / 1,000	$0.58 / 1,000	$0.57 / 1,000
Actor Start Infrequent	$0.001	$0.001	$0.001	$0.001
Platform usage
Platform costs	Included	Included	Included	Included

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.

Top Api

Website Content Crawler

jasondev/website-content-crawler

A powerful web crawler that extracts text content from websites, optimized for AI models, Large Language Models (LLMs), vector databases, and Retrieval-Augmented Generation (RAG) pipelines.

Jason Giang

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

Website Content Extractor for RAG: Markdown, HTML, Text

nezha/website-content-crawler

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.

nezha

Simple Website Scrapper (markdown format)

manojaditya64/simple-website-scrapper-markdown-format

A simple website scrapper that scrapes websites and converts it into markdown format which is easy to use with LLM. You can feed markdown data to LLM for easy analysis.

Manojaditya Nadar

5.0

(1)

AI-Ready Website Crawler

optimus-fulcria/ai-ready-website-crawler

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Fulcria Labs

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

Website & PDF to RAG JSONL Crawler

orbiscribe/linked-pdf-website-rag-crawler

Paste webpage and PDF URLs and get Markdown, JSONL chunks, PDF inventory, source warnings, and RAG-ready records.

Orbiscribe Labs

Website Main Content Extractor

sync-network/website-main-content-extractor

Extract the main content from any webpage — clean text without navigation, ads, footers, or boilerplate. Perfect for AI/LLM pipelines, RAG systems, content analysis, and data extraction. Returns title, description, author, publish date, and clean body text.