Pricing

from $2.00 / 1,000 page saveds

Try for free

Go to Apify Store

Website to Markdown Crawler

Try for free

Crawls a public website and saves clean Markdown for AI tools, docs and content audits. Same-site pages only, with robots.txt checked on every URL, heading-aware chunks, repeated boilerplate removed, and an optional change-only recrawl.

Pricing

from $2.00 / 1,000 page saveds

Rating

0.0

(0)

Developer

Vi Zellie

Actor stats

Bookmarked

Total users

Monthly active users

5 days ago

Last modified

Categories

Security

You can access the Website to Markdown Crawler programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "startUrls": ["https://docs.python.org/3/tutorial/index.html"],
10    "maxDepth": 1,
11    "maxPages": 10,
12    "includePatterns": ["/3/tutorial/*"],
13    "concurrency": 2,
14    "chunkSize": 500,
15    "chunkOverlap": 40,
16    "changesOnly": False,
17    "stateStoreName": "",
18    "extractPdfs": False,
19}
20
21# Run the Actor and wait for it to finish
22run = client.actor("vizellie/website-to-markdown-crawler").call(run_input=run_input)
23
24# Fetch and print Actor results from the run's dataset (if there are any)
25print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
26for item in client.dataset(run.default_dataset_id).iterate_items():
27    print(item)
28
29# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Website to Markdown Crawler API in Python

The Apify API client for Python is the official library that allows you to use Website to Markdown Crawler API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Website to Markdown Crawler API in JavaScript

Website to Markdown Crawler API through CLI

Website to Markdown Crawler OpenAPI definition

Website to Markdown Crawler API

🔥 FireScrape AI Website Content Markdown Scraper

mohamedgb00714/fireScraper-AI-Website-Content-Markdown-Scraper

Advanced web scraper powered by Crawlee and Puppeteer — extracts website content, converts it to Markdown, and structures it for LLM training datasets.

mohamed el hadi msaid

305

1.9

(2)

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.

Top Api

Website to Markdown Scraper for AI & LLMs

quaking_pail/ai-website-content-markdown-scraper

Crawl a website and get clean Markdown of each page's main content, with URL and title. Ready for LLMs, RAG, vector databases and AI agents. Headless Chrome, same-domain crawl, API access.

AI_Builder

946

3.5

(2)

Website Crawler API — Markdown for RAG

tugelbay/website-content-crawler

Website crawler API for public pages and clean Markdown, text, or HTML output for RAG pipelines, AI agents, documentation indexing, and monitoring. Guide: https://konabayev.com/tools/website-content-crawler/?utm_source=apify_info&utm_medium=referral&utm_campaign=website-content-crawler

Tugelbay Konabayev

Website & Page to Markdown for RAG and LLMs

humble-echidna/page-to-markdown

Turn web pages or whole sites into clean Markdown for RAG and LLMs: main content only, with title, links and word count. Whole-site mode reads the sitemap first, can return only changed pages, and splits pages into heading-based chunks. Respects robots.txt and AI opt-outs. No browser.

Michael Costa

Simple Website Scrapper (markdown format)

manojaditya64/simple-website-scrapper-markdown-format

A simple website scrapper that scrapes websites and converts it into markdown format which is easy to use with LLM. You can feed markdown data to LLM for easy analysis.

Manojaditya Nadar

5.0

(1)

Stealth Website Crawler

nocturne/stealth-website-crawler

Crawl websites protected by Cloudflare, DataDome, and other anti-bot systems. Extract clean text or markdown for AI/LLM pipelines. Like Website Content Crawler, but for sites that block you.

Nocturne

5.0

(1)

Website Content Crawler

alizarin_refrigerator-owner/website-crawler

Crawl websites for SEO audits. Extracts HTML, title, meta tags, headings, links, & text content from pages. Automatic sitemap detection & parsing Extracts metadata (title, description, OG tags) Heading structure (H1, H2, H3) Internal & external link analysis Image extraction w/alt text Word count

The Howlers

147

Website Content Extractor for RAG: Markdown, HTML, Text

nezha/website-content-crawler

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.