Pricing

from $3.00 / 1,000 converted markdown pages

Try for free

Go to Apify Store

AI-Ready Website to Clean Markdown Crawler (RAG & LLM)

Try for free

Crawls company websites via sitemap.xml or internal links, strips navigation, footers, scripts, and cookie banners, and converts main page content into clean, structured Markdown with metadata for LLMs and RAG pipelines.

Pricing

from $3.00 / 1,000 converted markdown pages

Rating

0.0

(0)

Developer

Ali J

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Why Use This Actor?

Raw HTML wastes up to 85% of LLM tokens on navigation menus, cookie banners, inline scripts, and SVG icons. This Actor automatically discovers pages via sitemap.xml or internal links, removes noise (nav, footer, script, cookie banners), and outputs structured Markdown with headings, bullet points, and word counts.

Zero Browser Overhead: Pure async httpx + lxml / BeautifulSoup4 for sub-second page conversion.
Pay-Per-Event (page-converted): Only pay for successfully cleaned and converted pages with meaningful text content.

Output Fields

Field	Type	Description
`title`	`string \| null`	Cleaned HTML `<title>` of the page
`url`	`string`	Canonical URL of the converted page
`markdown_content`	`string`	Structured Markdown (`#`, `##`, `-`, `>`) stripped of boilerplate
`word_count`	`integer`	Total word count of the extracted Markdown content

Input Example

{
  "urls": [
    { "url": "https://www.sipgate.de" }
  ],
  "maxPagesPerDomain": 5,
  "maxConcurrency": 10
}

Website to Markdown Converter & Crawler for LLMs and RAG

s3nafps/ai-ready-webpage-extractor

Website to Markdown Crawler is an Apify Actor that turns web pages, docs sites and blogs into clean Markdown and RAG chunks for LLMs. $3 per 1,000 pages.

mohamed senator

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

Website to Markdown Scraper for AI & LLMs

quaking_pail/ai-website-content-markdown-scraper

Crawl a website and get clean Markdown of each page's main content, with URL and title. Ready for LLMs, RAG, vector databases and AI agents. Headless Chrome, same-domain crawl, API access.

AI_Builder

946

3.5

(2)

Website & Page to Markdown for RAG and LLMs

humble-echidna/page-to-markdown

Turn web pages or whole sites into clean Markdown for RAG and LLMs: main content only, with title, links and word count. Whole-site mode reads the sitemap first, can return only changed pages, and splits pages into heading-based chunks. Respects robots.txt and AI opt-outs. No browser.

Michael Costa

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Locomint

AI-Ready Website Crawler

optimus-fulcria/ai-ready-website-crawler

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Fulcria Labs

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

halam

575

3.7

(3)

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Logiover

Html to Markdown Converter

antonio_espresso/html-to-markdown-converter

Crawl a target URL and convert its HTML content into clean, structured Markdown with optional heading-based chunking.

Antonio Blago

Web Page to Clean Markdown for RAG & LLM — 77% Fewer Tokens

craftbox/fast-markdown-for-rag

Convert web pages into clean Markdown for LLM and RAG pipelines, using 77% fewer tokens than a raw crawl while keeping every content section. Works in any language.