Pricing

from $3.00 / 1,000 page processeds

Website & Docs to Markdown + RAG Chunks

Turn websites & docs into clean Markdown plus token-bounded, embeddings-ready RAG chunks (heading lineage + sha256) ready for Pinecone, Weaviate, Qdrant or pgvector. Optional no-hallucination field extraction and AEO mode (FAQ, answer-first, llms.txt, citations). Robots honored; ownership required.

Pricing

from $3.00 / 1,000 page processeds

Rating

0.0

(0)

Developer

Adam

Actor stats

Bookmarked

Total users

Monthly active users

a month ago

Last modified

Categories

Developer tools

SEO tools

You can access the Website & Docs to Markdown + RAG Chunks programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "source": {
12        "mode": "url_list",
13        "urls": [
14            "https://example.com/"
15        ],
16        "maxPages": 1000
17    },
18    "ownership_attestation": true,
19    "chunking": {
20        "strategy": "semantic",
21        "maxTokens": 512,
22        "overlapTokens": 64
23    },
24    "extractSchema": {
25        "fields": [
26            {
27                "name": "product_name",
28                "type": "string",
29                "instruction": "The product name stated on the page; null if absent."
30            }
31        ]
32    },
33    "aeo": {
34        "assets": [
35            "faq_pair",
36            "answer_block",
37            "llms_txt"
38        ]
39    }
40};
41
42// Run the Actor and wait for it to finish
43const run = await client.actor("awesome_highboy/aeo-rag-ready-content-structurer").call(input);
44
45// Fetch and print Actor results from the run's dataset (if any)
46console.log('Results from dataset');
47console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
48const { items } = await client.dataset(run.defaultDatasetId).listItems();
49items.forEach((item) => {
50    console.dir(item);
51});
52
53// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Website to Markdown + RAG Chunks for Embeddings & LLMs API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Website & Docs to Markdown + RAG Chunks API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Website & Docs to Markdown + RAG Chunks API in Python

Website & Docs to Markdown + RAG Chunks API through CLI

Website & Docs to Markdown + RAG Chunks OpenAPI definition

Website & Docs to Markdown + RAG Chunks API

PDF to RAG Markdown Chunks for Embeddings

awesome_highboy/docforge

Convert PDFs into token-bounded Markdown chunks for RAG, embeddings, and vector databases (Pinecone, Chroma, Weaviate, Qdrant). Set maxTokens + overlap; get clean chunks with page number, token count, and SHA-256 content hash for dedup. JSON dataset ready for any LLM pipeline.

Adam

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

Website to Markdown for LLMs

agentictools/website-to-markdown-llm

Crawl a site and export clean Markdown with token counts and chunks, ready for RAG.

Ken Agland

Website to RAG Markdown Crawler

knotted_tussock/rag-markdown-crawler

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Ralph T

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Hitman studio

News to Markdown — RAG-Ready News Chunks API

nexgendata/news-announcements-rag-markdown

Convert news and announcements into RAG-ready Markdown chunks. Clean JSON for PR, media-monitoring teams and AI agents.

NexGenData

Website Content to Markdown (LLM-ready)

vivid_astronaut/website-content-to-markdown

Turn any website into clean, LLM-ready Markdown for RAG pipelines, AI agents and knowledge bases. Scrape single pages or crawl entire sites. Compliance-first: robots.txt honored.

BRAINIALL Team

RAG Web Crawler: Clean Markdown + Token-Sized Chunks

commonelements/rag-ready-crawler

Turn any website into embeddings-ready chunks for RAG and vector databases. Structure-aware token-sized chunking, clean LLM-ready markdown, per-chunk citations and metadata, dedup, and junk filtering. Pay per result, no surprise compute bills.

Harry Schoeller

Docs to Markdown + AI Embeddings → Vector DB Crawler

badruddeen/docs-to-markdown-ai-embeddings---vector-db-crawler

Turn any documentation site into clean Markdown, intelligently chunked content with embeddings (Azure/OpenAI), and directly upsert into MongoDB Atlas, Pinecone, Weaviate, Qdrant, or Milvus — ready for RAG, AI assistants, and semantic search in minutes.

Badruddeen Naseem

5.0

Website to RAG Dataset

sebastian-actors/website-to-rag-dataset

Convert public websites, docs, blogs, and XML sitemaps into clean Markdown, structured metadata, and stable chunks for RAG pipelines and vector databases.