Website to Markdown Crawler

Scrape job postings from Computrabajo (14 countries), Bumeran (AR, PE), Multitrabajos (EC) and Konzerta (PA) in one run. Keyword and location search, salary, company, remote, date. Normalized JSON.

Llm Ready Documentation Scraper

direct_duty/llm-ready-documentation-scraper

Developers and AI agents need to read documentation (e.g. Stripe Docs, Next.js Docs), but standard scrapers return noisy HTML that includes: navigation bars headers / footers ads / cookie banners This Actor must return pure content-only Markdown, suitable for vectorization and semantic search.

A powerful web crawler that extracts text content from websites, optimized for AI models, Large Language Models (LLMs), vector databases, and Retrieval-Augmented Generation (RAG) pipelines.

Crawl any website and get each page's main content as clean Markdown and text for LLMs, RAG and search. Sitemap-seeded, de-duplicated, no nav or cookie banners. Respects robots.txt and AI-crawler opt-outs. $1.50 per 1,000 pages.

Markdown Maker: HTML to Markdown 📝

shahidirfan/Markdown-Maker

Instantly convert complex HTML into clean, structured Markdown. This lightweight actor is optimized to render web content into a format that is easily readable for AI LLMs, reducing token usage and improving context. Perfect for RAG pipelines and preparing data for training.

Website Content Crawler (Anti-Bot) - $1.20 / 1000 Pages

scrapeunblocker/website-content-crawler

Crawl any website, even behind Cloudflare, DataDome, PerimeterX or Akamai, and get clean Markdown and text of every page for LLMs, RAG and AI agents. Main-content extraction, URL globs, sitemaps, depth and page limits. $1.20 per 1000 pages; failed pages are free.

Web Fetch API — Fetch Any URL as HTML, Text, Markdown or JSON

highbrow_fame/web-fetch-api

Bulk HTTP fetch for agents: returns status, headers and body (raw HTML, text, Markdown or parsed JSON), follows redirects, optional browser rendering. Unofficial Web Fetch API.

Stealth Website Crawler

nocturne/stealth-website-crawler

Crawl websites protected by Cloudflare, DataDome, and other anti-bot systems. Extract clean text or markdown for AI/LLM pipelines. Like Website Content Crawler, but for sites that block you.

Scrape any website to clean, LLM-ready markdown — a compliant Firecrawl alternative for RAG ingestion, robots.txt always on.

Every Wayback Machine (archive.org) capture of a URL, folder, host or domain: timestamp, status, MIME type, digest and archive links. Unique versions, daily/yearly sampling, unique URLs, per-site summaries, optional archived page Markdown. $2 per 1,000 snapshots.