Pricing

from $0.001 / result

Try for free

Go to Apify Store

Website Content Extractor for RAG: Markdown, HTML, Text

Try for free

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.

Pricing

from $0.001 / result

Rating

5.0

(2)

Developer

nezha

Actor stats

Bookmarked

Total users

Monthly active users

17 days ago

Last modified

Categories

Developer tools

SEO tools

You can access the Website Content Extractor for RAG: Markdown, HTML, Text programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "startUrls": [
12        {
13            "url": "https://docs.apify.com/academy"
14        }
15    ],
16    "sitemapUrls": [
17        "https://docs.apify.com/sitemap.xml"
18    ],
19    "excludeUrlGlobs": [
20        "**/search/**",
21        "**/login/**"
22    ]
23};
24
25// Run the Actor and wait for it to finish
26const run = await client.actor("nezha/website-content-crawler").call(input);
27
28// Fetch and print Actor results from the run's dataset (if any)
29console.log('Results from dataset');
30console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
31const { items } = await client.dataset(run.defaultDatasetId).listItems();
32items.forEach((item) => {
33    console.dir(item);
34});
35
36// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Website Content Extractor for RAG: Markdown, HTML, Text API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Website Content Extractor for RAG: Markdown, HTML, Text API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Website Content Extractor for RAG: Markdown, HTML, Text API in Python

Website Content Extractor for RAG: Markdown, HTML, Text API through CLI

Website Content Extractor for RAG: Markdown, HTML, Text OpenAPI definition

Website Content Extractor for RAG: Markdown, HTML, Text API

Website to Markdown and RAG Dataset Crawler

orbiscribe/website-rag-dataset-builder

Crawl public websites into clean Markdown, text, metadata, links, JSON-LD, and chunks for RAG and knowledge bases.

Orbiscribe Labs

Website Content Extractor

taroyamada/website-content-extractor

Extract clean text and markdown from docs, pricing, product, policy, and help-center URLs for RAG datasets and content operations.

太郎山田

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

Website Content Crawler

crawlerbros/website-content-crawler

Crawls websites and extracts clean text, markdown, or HTML content. Ideal for LLM training data, RAG pipelines, and knowledge base building.

Crawler Bros

5.0

Website Content Crawler

parseforge/website-content-crawler

Crawl any website and pull clean Markdown content ready for AI! Follow links across a whole domain and extract page text, titles, headings, images, and metadata. Perfect for building RAG pipelines, training datasets, knowledge bases, and vector databases. Start crawling content in minutes!

ParseForge

Website to Markdown — Clean Pages for RAG & LLMs

ryanclinton/website-content-to-markdown

Crawl any website and convert its HTML pages into clean, well-structured Markdown text. Purpose-built for AI workflows, RAG pipelines, LLM fine-tuning, and documentation archival. Give it URLs and get back Markdown documents with navigation, ads, and boilerplate stripped away.

Ryan Clinton

Website Content Crawler API - Markdown for RAG

tugelbay/website-content-crawler

Crawl public websites and extract clean Markdown, text, or HTML for RAG pipelines, AI agents, documentation indexing, and content monitoring. Guide: https://konabayev.com/tools/website-content-crawler/?utm_source=apify_info&utm_medium=referral&utm_campaign=website-content-crawler

Tugelbay Konabayev

Website to Markdown for LLM and RAG

jeweled_jockstrap/my-actor-3

Convert any URL to clean Markdown text for AI applications. Strips HTML extracts content. For LLM training RAG pipelines and vector databases. Free Firecrawl alternative.