Pricing

Pay per usage

Go to Apify Store

Website Content RAG Cleaner

Try for free

Extract public web pages into clean, structured content ready for retrieval, summaries, and AI workflows.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Daniel Ferreira de Almeida

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

What it does

This Actor fetches one or more public URLs, or consumes provided HTML, and returns:

the resolved page title and final URL
cleaned plain text
markdown-style content
optional raw HTML
extracted links
chunked text for retrieval pipelines
basic run metadata

Best for

knowledge base ingestion
RAG pipelines
AI search indexes
support and sales enablement
document-style content extraction

Input

urls — public URLs to process
html — optional single HTML document for offline validation
htmlByUrl — optional offline URL→HTML map
includeRawHtml — include the original HTML in output
includeLinks — include extracted links
maxTextChars — cap on cleaned text length
chunkSize — chunk size for downstream processing

Example input

{
  "urls": ["https://example.com/article"],
  "html": "<!doctype html><html><head><title>Example</title></head><body><h1>Example</h1><p>Hello world.</p></body></html>",
  "includeRawHtml": false,
  "includeLinks": true,
  "maxTextChars": 5000,
  "chunkSize": 400
}

Output

Each item contains:

url
finalUrl
title
description
text
markdown
chunks
links
wordCount
status
checkedAt

Validation

The local smoke test uses a fixture HTML document so the Actor can be validated without network dependence.

Notes

Safe for platform testing with fixture HTML.
Designed for clean ingestion into downstream retrieval systems.

Refinery — HTML to LLM Text (Cut RAG Token Cost)

larelabs/refinery-html-to-llm-cleaner

HTML to LLM text cleaner for RAG pipelines. Strip scripts, nav & layout junk after Firecrawl or your fetch. BeautifulSoup-alternative speed. $0.002/page · 3 README demos in Console.

Lare Labs

Website Content Extractor for RAG: Markdown, HTML, Text

nezha/website-content-crawler

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.

nezha

5.0

(2)

RAG Post Processor - Text Cleaner & Chunker for LLM Pipelines

jalicia/rag-post-processor

Clean and chunk scraped text for RAG and LLM pipelines. Strips HTML, collapses whitespace, splits into overlapping chunks ready for embedding. Works standalone or chained after any scraper. Per-row billing.

Jordan Wagner

Website Content Miner

seeb/website-content-miner

Extract clean website content at scale: page titles, meta descriptions, H1-H3 headings, readable main text, and URLs. Includes smart noise removal, Readability fallback, optional internal crawling, and structured output for SEO audits, AI datasets, research, and automation.

Techionik

5.0

(1)

Website Content Crawler

parseforge/website-content-crawler

Crawl any website and pull clean Markdown content ready for AI! Follow links across a whole domain and extract page text, titles, headings, images, and metadata. Perfect for building RAG pipelines, training datasets, knowledge bases, and vector databases. Start crawling content in minutes!

ParseForge

RAG Web Browser

parseforge/rag-web-browser

Give your AI agents real-time web access! Search the web on any topic and get full page content as clean Markdown, ready for LLMs, RAG pipelines, or OpenAI Assistants. Includes titles, descriptions, links, authors, images, and metadata. Start grounding your AI with fresh data in minutes!

ParseForge

🌐 Website Content Crawler

citrine_venus/website-content-crawler

Crawl any website into clean text, Markdown & rich metadata — AI-ready output for RAG, LLMs & vector databases. Streams to your dataset in real time.

Data Minds

Article Scraper & News Scraper API

tugelbay/article-extractor

Article scraper API for clean text and metadata from URLs as Markdown, text, or HTML for RAG, AI agents, monitoring, and research. Guide: https://konabayev.com/tools/article-extractor/?utm_source=apify_info&utm_medium=referral&utm_campaign=article-extractor

Tugelbay Konabayev

RAG Web Browser

scraper-engine/rag-web-browser

RAG Web Browser websites and extract relevant content for Retrieval-Augmented Generation (RAG). Collect clean text, metadata, links, and documents from web pages, then export structured data to JSON, CSV, Excel, or XML for AI assistants, knowledge bases, and semantic search.

Scraper Engine

RAG Spider - Web to Markdown Crawler for AI Training Data

lenient_grove/RAG-Spider

Enterprise-grade web crawler that converts messy websites into clean, chunked Markdown for AI systems. Uses Mozilla Readability for 95% cleaner extraction than competitors. Outputs RAG-ready data with metadata and token estimates. Perfect for building knowledge bases and training AI chatbots.