Pricing

Pay per usage

Web to Markdown — AI-Ready Text from Any URL

Convert any web page URL to clean Markdown format. Perfect for LLM training data, RAG pipelines, and AI content processing. Extracts main content, strips ads/nav/footers.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

陈俊杰

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

🌐 Web to Markdown Converter — Apify Actor

将任意网页URL转换为干净的Markdown格式，专为AI/LLM数据处理场景设计。

📋 功能简介

一键抓取：输入URL，自动获取网页HTML
智能提取：自动识别并提取主体内容（文章/主要内容区块），去除广告、导航栏、页脚、侧边栏等干扰元素
干净输出：使用 markdownify 将HTML转换为标准Markdown格式
可选的CSS选择器：指定特定区域进行提取
错误处理完备：HTTP错误、超时、解析异常均有妥善处理

🎯 适用场景

场景	说明
LLM训练数据准备	将网页内容转为结构化文本供大模型训练
RAG流水线	网页文档 → 向量数据库的预处理步骤
AI内容处理	配合LLM进行摘要、翻译、分析等工作流
数据归档	将在线文章保存为可读的纯文本格式
网页内容对比	提取不同版本的页面文本进行差异分析

📥 输入参数

参数	类型	必填	默认值	说明
`url`	string	✅	—	目标网页URL
`selector`	string	❌	`null`	CSS选择器，指定提取的区域（如 `.article-body`）
`include_images`	boolean	❌	`false`	是否在Markdown中包含图片链接

📤 输出字段

字段	类型	说明
`url`	string	源网页URL
`title`	string	页面标题
`markdown`	string	转换后的Markdown文本
`word_count`	integer	Markdown的单词数量
`char_count`	integer	Markdown的字符数量
`extracted_at`	string	提取时间（UTC ISO 8601）
`error`	string	处理失败时的错误信息

🚀 快速使用

通过Apify平台

打开 Web to Markdown Converter Actor页面
点击 Run
输入目标URL，点击 Start
获取Markdown输出

通过Apify API

import requests

response = requests.post(
    "https://api.apify.com/v2/acts/<username>~web-to-markdown/runs",
    json={
        "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
        "include_images": False
    }
)
print(response.json())

通过Apify SDK (Python)

from apify import Actor

async def main():
    async with Actor:
        run_input = {
            "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
            "include_images": False
        }
        run = await Actor.call(
            "username/web-to-markdown",
            run_input=run_input
        )
        dataset = await run.dataset.get_items()
        print(dataset[0]["markdown"][:500])

🛠 本地开发

前置条件

Python 3.14+
Apify CLI (npm install -g apify-cli)

本地运行

# 安装依赖
pip install -r requirements.txt

# 通过Apify CLI运行
apify run

# 或直接运行Python
python -m src

测试

# 设置环境变量
export APIFY_LOCAL_STORAGE_DIR=./apify_storage

# 运行
apify run

📦 技术栈

Apify SDK (Python) — Actor框架
httpx — 异步HTTP客户端
BeautifulSoup4 — HTML解析
markdownify — HTML→Markdown转换

📄 许可证

MIT

AI Web to Markdown - LLM-Ready Extractor

wiry_kingdom/ai-web-to-markdown

Convert any URL into clean LLM-ready markdown. Strips ads, nav, footer. Preserves headings, lists, tables, code blocks. Returns token count. Perfect for RAG, fine-tuning, AI agents. 10x cheaper than Firecrawl.

Mohieldin Mohamed

Rag Content Chunker

labrat011/rag-content-chunker

Turn raw text, Markdown, or Apify datasets into token-perfect RAG chunks with deterministic IDs, source metadata, and a billing-ready summary—ready for embeddings or vector DBs without extra glue code.

mick_

Website to Clean Markdown (AI & RAG Ready)

ahmed_jasarevic/website-to-clean-markdown-ai-rag-ready

Convert any website into clean, noise-free Markdown. Perfect for training LLMs, building Custom GPTs, and RAG pipelines. Save 80% on OpenAI tokens by stripping HTML junk.

Ahmed Jasarevic

Markitdown Mcp Server

rector_labs/markitdown-mcp-server

Cloud-hosted MCP server converting 29+ document formats (PDF, DOCX, PPTX, images, audio) to AI-ready Markdown. Zero Python setup. Perfect for RAG pipelines and AI agents. Pay-per-use: $0.02/conversion. Built on Microsoft's Markitdown (82k+ ⭐).

RECTOR SOL

AI Markdown Maker

onescales/bulk-ai-markdown-maker

Convert any web page into clean, AI ready markdown format in seconds. This markdown generator is perfect for content for AI models, creating documentation, or archiving web content. It intelligently parses web content, removing ads, navigation, and other clutter. Generate Markdown Today!

One Scales

127

5.0

(2)

AI Dataset Converter - Website to Training Data

boztek-ltd/ai-dataset-converter

Crawl websites and convert content into AI-ready formats: RAG chunks, fine-tuning JSONL, Q&A pairs, clean Markdown. Token-aware chunking, quality scoring, deduplication. No external LLM API needed.

Boztek LTD

Reddit Scraper — Posts, Comments & Markdown for AI/RAG

benthepythondev/reddit-scraper

Extract Reddit posts, comments & user data in AI-ready markdown format. No API keys needed! 25% cheaper than competitors. Perfect for AI training, sentiment analysis & market research. Includes bulk comment scraping with progress tracking.

ben

5.0

(1)

YouTube Transcript API | Video to Text Scraper for AI

andok/youtube-transcript-scraper

Extract full transcripts and time-coded captions from any YouTube video. Build custom AI datasets, train LLMs, or repurpose video content.

Andok

Article to Text Extractor (for TTS/LLMs)

andok/tts-reader

Extract the core readable text of any article or blog post, stripping out boilerplate. Perfect for Text-to-Speech or AI summaries.

Andok

AI-Powered Smart Web Scraper

cloud9_ai/ai-web-scraper

Intelligent content extraction from any website using Crawlee + AI. Auto-detects structure, adapts to layout changes, handles JavaScript rendering. No custom code needed. Extract articles, products, listings from 1000s of pages.