Pricing

from $3.00 / 1,000 page extracteds

LLM-Ready Web Content Extractor

Turn any public web page or shallow site crawl into clean, LLM-ready Markdown + metadata + optional RAG chunks. Boilerplate/nav/ads removed. Honors robots.txt.

Pricing

from $3.00 / 1,000 page extracteds

Rating

0.0

(0)

Developer

Leopold Ai

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Why this Actor

Raw HTML is noisy — nav bars, cookie banners, ads, footers. This Actor pulls out just the main content, converts it to clean Markdown, and hands you structured, ready-to-use output. No parsing, no cleanup, no LLM tokens wasted on junk.

What you get (per page)

Field	Description
`markdown`	Clean main content as Markdown (links & tables preserved)
`text`	Plain-text version
`title`, `description`, `author`, `date`, `siteName`	Page metadata
`wordCount`, `tokenEstimate`	Size at a glance
`chunks`	Optional overlapping chunks, ready for a vector DB

Example

Input

{
  "startUrls": [{ "url": "https://en.wikipedia.org/wiki/Vector_database" }],
  "maxPages": 1,
  "ragChunking": true,
  "chunkWords": 200
}

Output (excerpt)

{
  "url": "https://en.wikipedia.org/wiki/Vector_database",
  "title": "Vector database - Wikipedia",
  "wordCount": 1528,
  "tokenEstimate": 3121,
  "markdown": "| Part of a series on | Machine learning and data mining | ...",
  "chunks": ["Vector database — a database that stores ...", "..."]
}

Use cases

RAG ingestion — clean text + chunks straight into your vector DB
Agent tools — give an agent a URL, get usable content back
Content pipelines — docs, blogs, news, wikis → structured Markdown
Research & analysis — summarize or analyze clean page content

Inputs

startUrls — pages to extract (seeds when crawling)
maxPages, maxCrawlDepth, sameDomainOnly — control scope & cost
renderJs — render JavaScript with a headless browser (for client-rendered SPAs)
ragChunking, chunkWords, chunkOverlapWords — RAG output
respectRobots (default on), includeHtml, contactEmail

Two modes

HTTP (default) — fast & cheap; great for server-rendered pages (blogs, news, docs, wikis — most content sites).
renderJs — headless Chromium renders the page first, so JavaScript-heavy SPAs (client-rendered apps, some docs) work too. Slower; turn on only when needed.

Responsible by design

Fetches only public pages, honors robots.txt, identifies itself in the User-Agent, and never touches login/paywalled content or personal data. Use it on content you're permitted to process.

Pricing

Pay per page extracted — you're charged only for pages that succeed. Skipped, blocked, or failed pages are free.

AI Training Data Scraper - LLM and RAG-Ready

george.the.developer/ai-training-data-scraper

Extract web content formatted for LLM fine-tuning and RAG pipelines. Output in OpenAI JSONL, Claude JSONL, Markdown, or raw text.

George Kioko

AI-Ready Web Content Crawler (LLM/RAG Optimized)

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Yuliia Kulakova

AI Web Crawler

darknezz/ai-web-crawler

Extract clean, chunked content from any website for LLMs and AI agents. Gets title, headings, main content, links, tables, and JSON-LD. Outputs token-estimated chunks ready for RAG pipelines.

Oaida Adrian

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

Website Content Crawler Pro — Markdown for AI & RAG

hipersoft/web-content-crawler-pro

Crawl any website into clean Markdown or text for LLMs, RAG and AI. HTTP + JS-browser modes, main-content (readability) extraction, sitemaps, URL glob filters, cookie-banner removal and CSS selectors. No login, no API key.

hiper soft

Article Scraper & News Scraper API

tugelbay/article-extractor

Article scraper API for clean text and metadata from URLs as Markdown, text, or HTML for RAG, AI agents, monitoring, and research. Guide: https://konabayev.com/tools/article-extractor/?utm_source=apify_info&utm_medium=referral&utm_campaign=article-extractor

Tugelbay Konabayev

Quick Website Content Scraper ( Extract Text for RAG & LLMs )

automateitplease/ai-web-content-scraper-extract-text-for-rag-llms

Extract clean text from any website for AI/LLM applications. Supports both static and JavaScript-rendered sites (React, Vue, Angular). Perfect for RAG systems, chatbot training, and content analysis.

AutomateItPlease Workflow And Automaton Ops

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

AI Content Detector 🔍

easyapi/ai-content-detector

🤖 Analyze text content to determine if it's AI-generated with high accuracy. Get detailed probability analysis and authoritative conclusions about content authenticity. Perfect for content verification, academic integrity, and digital publishing quality control.

EasyApi

Website to Markdown Converter

lofomachines/website-to-markdown-converter

Best faster and cheaper way to convert any web page into clean, structured, LLM-ready Markdown.

Lofomachines

Article Content Extractor 📄

easyapi/article-content-extractor

Extract clean article content, metadata and structured information from any web page. Supports multiple URLs and returns well-formatted JSON with title, description, content, author, publish date and more. 🔍📄

EasyApi

147

5.0

(1)