Pricing

Pay per usage

Website to Markdown - Clean LLM-Ready Content

Convert any webpage into clean markdown stripped of navigation, ads, and boilerplate. Perfect for RAG pipelines, LLM context, and content extraction. Token counts included.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

C. K.

Actor stats

Bookmarked

Total users

Monthly active users

5 hours ago

Last modified

Website to Markdown — Clean, LLM-Ready Content Extraction

Convert any webpage or website into clean markdown, stripped of navigation, ads, sidebars, and boilerplate. Output drops straight into any RAG pipeline, LLM context window, or vector store without cleanup. Token counts included so you can plan your embedding budget.

What it does

Most web scrapers give you raw HTML or a wall of unstructured text. You then spend hours cleaning, reformatting, and fixing broken context. This Actor eliminates that step.

Give it a URL. It crawls the site, strips all chrome (navigation, sidebars, footers, cookie banners), and converts each page to clean markdown preserving headings, code blocks, tables, lists, and links. Every page includes a token count (cl100k_base encoding) so you know exactly what it costs to embed or send to an LLM.

Output format

Field	Type	Description
`url`	string	Source URL of the page
`title`	string	Page title
`content`	string	Clean markdown content
`token_count`	integer	Token count (cl100k_base encoding)
`content_length`	integer	Character count
`meta_description`	string	Page meta description (if available)

Input parameters

Parameter	Type	Default	Description
`startUrl`	string	—	URL to start crawling from
`urls`	array	—	List of specific URLs to convert (batch mode)
`maxPages`	integer	`50`	Maximum pages to convert
`crawlSameDomain`	boolean	`true`	Stay within the start URL's domain
`pathPrefix`	string	`""`	Only crawl paths starting with this prefix
`outputFormat`	string	`"markdown"`	`"markdown"` or `"plain_text"`
`includeMetadata`	boolean	`true`	Include token count and meta description

Example usage

Single page

{
    "startUrl": "https://docs.python.org/3/library/asyncio.html",
    "maxPages": 1
}

Batch conversion

{
    "urls": [
        "https://example.com/page-1",
        "https://example.com/page-2",
        "https://example.com/page-3"
    ],
    "maxPages": 3
}

Full site crawl

{
    "startUrl": "https://fastapi.tiangolo.com/",
    "maxPages": 100,
    "pathPrefix": "/tutorial/"
}

Pricing

This Actor uses the pay-per-event model. You are charged per page successfully converted to markdown. No charge for pages that are skipped (empty, non-content).

How it works

Crawl — Crawlee handles the URL queue, deduplication, rate limiting, and robots.txt compliance.
Clean — Strips navigation, sidebars, footers, cookie banners, and boilerplate using curated selectors. Falls back to <article>, <main>, or <body>.
Convert — Transforms clean HTML to structured markdown, preserving headings, code blocks, tables, lists, and links.
Count — Uses cl100k_base (GPT-4 / modern embedding encoding) for accurate token counts.

Responsible use

This Actor respects robots.txt by default (enforced by Crawlee).
Crawlee's built-in autoscaling keeps request rates reasonable.
You are responsible for ensuring your use complies with the target site's Terms of Service.

Built with

Crawlee for reliable crawling
BeautifulSoup for HTML parsing
tiktoken for token counting

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

AI Web to Markdown - LLM-Ready Extractor

wiry_kingdom/ai-web-to-markdown

Convert any URL into clean LLM-ready markdown. Strips ads, nav, footer. Preserves headings, lists, tables, code blocks. Returns token count. Perfect for RAG, fine-tuning, AI agents. 10x cheaper than Firecrawl.

Mohieldin Mohamed

Firecrawl AI-Powered Web Search & Scrape

alizarin_refrigerator-owner/firecrawl-ai-powered-web-search-scrape

Search the web and get clean, LLM-ready content in one API call. Powered by Firecrawl's /v1/search endpoint. Returns markdown, HTML, or extracted data. Perfect for SEO research, competitor analysis, and AI training data collection.

The Howlers

Website SEO & Metadata Checker — Meta, OpenGraph, JSON-LD

dataquarry/website-seo-metadata-checker

Free SEO & metadata audit for any URL: title, meta description, OpenGraph, Twitter card, JSON-LD types, canonical, hreflang, viewport, robots, and link/image checks. No API key.

Daniel Brenner

Site to Markdown — any site to clean, LLM-ready markdown

topsail/site-to-markdown

Scrape any website to clean, LLM-ready markdown — a compliant Firecrawl alternative for RAG ingestion, robots.txt always on.

Connor Teskey

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

321

5.0

(2)

AI-Ready Web Content Crawler (LLM/RAG Optimized)

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Yuliia Kulakova

Website To Markdown

smart_api/website-to-markdown

Convert any webpage into clean, LLM-ready Markdown in seconds — perfect for AI training data, RAG pipelines, and content archiving.

SmartApi

5.0

(1)

Udemy Course Scraper 📚

easyapi/udemy-course-scraper

Extract detailed course information from Udemy.com with this powerful scraper. Collect comprehensive data about online courses, including ratings, content details, instructors, and pricing. Perfect for market research, content aggregation, and educational platform development.

EasyApi

5.0

(1)

Article Content Extractor 📄

easyapi/article-content-extractor

Extract clean article content, metadata and structured information from any web page. Supports multiple URLs and returns well-formatted JSON with title, description, content, author, publish date and more. 🔍📄

EasyApi

130

ExpiredDomains.net Scraper 🔍

easyapi/expireddomains-net-scraper

Scrape expired domains data from ExpiredDomains.net. Extract detailed domain information including domain status, backlinks, creation date, and availability across multiple TLDs.

EasyApi

258

1.8

(4)