Pricing

Pay per event

Try for free

Go to Apify Store

Website to Markdown Crawler

Try for free

Crawl a website from start URLs within the same domain and get clean Markdown, title and word count per page. Respects robots.txt. Optional JS rendering. Flat, predictable per-page pricing.

Pricing

Pay per event

Rating

0.0

(0)

Developer

Small Gears Co

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

What it does

Follows links only within the same domain as each start URL (optionally only under the start path).
Respects robots.txt, including crawl-delay.
Stops at your max-pages limit, so the cost of a run is known in advance.
Strips navigation, headers, footers, scripts and forms, then converts the main content to Markdown.
Optional JavaScript rendering with Playwright (off by default to keep runs cheap and fast).

Input

See .actor/input_schema.json. Example:

{
  "startUrls": ["https://docs.apify.com/academy"],
  "maxPages": 5,
  "stayUnderStartPath": true
}

Output

One dataset item per page:

{ "url": "...", "title": "...", "markdown": "# Heading ...", "wordCount": 812, "crawledAt": "2026-10-15T17:00:00.000Z" }

Pricing

Planned flat per-page pricing (for example $1 per 1,000 pages). Not yet enabled.

Local test

npm install
mkdir -p storage/key_value_stores/default && cp test-input.json storage/key_value_stores/default/INPUT.json
npm start

About

Built and maintained by Small Gears Co (https://smallgearsco.com). Questions or feature requests: contact@smallgearsco.com

LatAm Jobs Scraper: Computrabajo, Bumeran, Multitrabajos

datadesk-io/latam-jobs-scraper

Scrape job postings from Computrabajo (14 countries), Bumeran (AR, PE), Multitrabajos (EC) and Konzerta (PA) in one run. Keyword and location search, salary, company, remote, date. Normalized JSON.

DataDesk

Llm Ready Documentation Scraper

direct_duty/llm-ready-documentation-scraper

Developers and AI agents need to read documentation (e.g. Stripe Docs, Next.js Docs), but standard scrapers return noisy HTML that includes: navigation bars headers / footers ads / cookie banners This Actor must return pure content-only Markdown, suitable for vectorization and semantic search.

Sean

Website Content Crawler

jasondev/website-content-crawler

A powerful web crawler that extracts text content from websites, optimized for AI models, Large Language Models (LLMs), vector databases, and Retrieval-Augmented Generation (RAG) pipelines.

Jason Giang

Website Content Crawler: Markdown for AI (Half Price)

laudable_instrument/website-content-crawler-half-price

Crawl any website and get each page's main content as clean Markdown and text for LLMs, RAG and search. Sitemap-seeded, de-duplicated, no nav or cookie banners. Respects robots.txt and AI-crawler opt-outs. $1.50 per 1,000 pages.

Unadhi

Markdown Maker: HTML to Markdown 📝

shahidirfan/Markdown-Maker

Instantly convert complex HTML into clean, structured Markdown. This lightweight actor is optimized to render web content into a format that is easily readable for AI LLMs, reducing token usage and improving context. Perfect for RAG pipelines and preparing data for training.

Shahid Irfan

5.0

(1)

Website Content Crawler (Anti-Bot) - $1.20 / 1000 Pages

scrapeunblocker/website-content-crawler

Crawl any website, even behind Cloudflare, DataDome, PerimeterX or Akamai, and get clean Markdown and text of every page for LLMs, RAG and AI agents. Main-content extraction, URL globs, sitemaps, depth and page limits. $1.20 per 1000 pages; failed pages are free.

Scrapeunblocker

Web Fetch API — Fetch Any URL as HTML, Text, Markdown or JSON

highbrow_fame/web-fetch-api

Bulk HTTP fetch for agents: returns status, headers and body (raw HTML, text, Markdown or parsed JSON), follows redirects, optional browser rendering. Unofficial Web Fetch API.

yestrue

Stealth Website Crawler

nocturne/stealth-website-crawler

Crawl websites protected by Cloudflare, DataDome, and other anti-bot systems. Extract clean text or markdown for AI/LLM pipelines. Like Website Content Crawler, but for sites that block you.

Nocturne

5.0

(1)

Site to Markdown — any site to clean, LLM-ready markdown

topsail/site-to-markdown

Scrape any website to clean, LLM-ready markdown — a compliant Firecrawl alternative for RAG ingestion, robots.txt always on.

Connor Teskey

Wayback Machine Scraper — Archive.org Snapshots & URL History

actor-forge/wayback-snapshots

Every Wayback Machine (archive.org) capture of a URL, folder, host or domain: timestamp, status, MIME type, digest and archive links. Unique versions, daily/yearly sampling, unique URLs, per-site summaries, optional archived page Markdown. $2 per 1,000 snapshots.