Blog Post Scraper for LLM

AI-Ready Web Content Crawler (LLM/RAG Optimized)

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Content Gap Analyzer

fabian_maume/content-gap-analyzer

Identifies topics, insights, and questions that top-ranking competitors cover but your blog post misses. Runs a Google search, classifies each competitor page, extracts key insights via LLM, and reports what's missing from your own article.

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

Sitemap URL Extractor

crawlerbros/sitemap-url-extractor

Extract every URL from any site's sitemap.xml with handles sitemap index files (nested sitemaps), gzipped sitemaps, and robots.txt discovery. Returns URL, lastmod, changefreq, priority, and optional image/video/alternate-language fields. No proxy, no cookies, no login.

LLM Markdown Crawler

sleek_waveform/llm-markdown-crawler

Crawl any website and extract clean, boilerplate-free Markdown optimized for LLMs, RAG pipelines, and AI training datasets. Uses Mozilla Readability to strip navigation and ads, then converts to clean Markdown. No browser required — fast and cheap.

Broken Link Checker — Recursive Site Crawler

accurate_pouch/broken-link-checker

Recursively crawl your website and find every broken link, 404, redirect, and timeout. Checks internal and external links with configurable depth. 100 links free per run.

Website Change Monitor — Content Diff & Alerts

accurate_pouch/website-change-monitor

Monitor websites for content changes. Text diff, hash comparison, CSS selector targeting. Webhook alerts when content changes. Use with Apify scheduling for daily monitoring. 5 URLs free.

Extract the core readable text of any article or blog post, stripping out boilerplate. Perfect for Text-to-Speech or AI summaries.

Convert any URL into clean LLM-ready markdown. Strips ads, nav, footer. Preserves headings, lists, tables, code blocks. Returns token count. Perfect for RAG, fine-tuning, AI agents. 10x cheaper than Firecrawl.

🔥 AI HTML to JSON Extractor (Fast, Free LLM for Data)

autoscaler/ai-html-to-json-extractor

Eliminate messy HTML cleanup and high LLM costs. This Actor uses a high-speed, zero-cost large language model to turn unstructured content (HTML, text, reviews, blog posts) into valid, structured JSON.