Pricing

Pay per usage

RAG Website Crawler — Chunked Markdown for LLMs ✅

Affordable RAG website crawler: clean Markdown for LLMs & RAG. Free (compute-only), no per-result charge, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

Pricing

Pay per usage

Rating

0.0

(0)

Developer

The Mine Works

Actor stats

Bookmarked

Total users

Monthly active users

7 hours ago

Last modified

Categories

Developer tools

MCP servers

You can access the RAG Website Crawler — Chunked Markdown for LLMs ✅ programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "startUrls": [{ "url": "https://example.com" }],
10    "excludeUrlPatterns": [
11        "\\.(pdf|zip|png|jpg|jpeg|gif|svg|mp4|mp3|woff2?)$",
12        "/tag/",
13        "/author/",
14    ],
15}
16
17# Run the Actor and wait for it to finish
18run = client.actor("themineworks/rag-crawler").call(run_input=run_input)
19
20# Fetch and print Actor results from the run's dataset (if there are any)
21print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
22for item in client.dataset(run["defaultDatasetId"]).iterate_items():
23    print(item)
24
25# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

RAG Website Crawler — Chunked Markdown for LLMs ✅ API in Python

The Apify API client for Python is the official library that allows you to use RAG Website Crawler — Chunked Markdown for LLMs ✅ API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

RAG Website Crawler — Chunked Markdown for LLMs ✅ API in JavaScript

RAG Website Crawler — Chunked Markdown for LLMs ✅ API through CLI

RAG Website Crawler — Chunked Markdown for LLMs ✅ OpenAPI definition

RAG Website Crawler — Chunked Markdown for LLMs ✅ API

Threads Scraper — Posts, Profiles & Hashtags ✅ No Login

themineworks/threads-scraper

Affordable Threads scraper: posts, profiles & replies. $2/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

LinkedIn Employees Scraper — B2B Leads & Profiles ✅ No Cookies

themineworks/linkedin-employees

Affordable LinkedIn employees scraper: B2B leads, no login. $2/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

OpenAlex Scraper — Works, Authors & Citations ✅

themineworks/openalex-scholarly-works

Affordable OpenAlex scraper: scholarly works, authors & citations. $2/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

Google News Scraper — Headlines, Stories & Topics ✅ No API Key

themineworks/google-news

Affordable Google News scraper: headlines, sources & topics. $3/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

AliExpress Scraper — Products, Prices & Ratings ✅ No Login

themineworks/aliexpress-products

Cheapest AliExpress product scraper: titles, prices, ratings & orders. $2.50/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

Reddit Scraper — Posts, Comments & Subreddits ✅ No Login

themineworks/reddit-scraper

Cheapest Reddit scraper: posts, comments, scores & subreddits. 100% free, no subscription, no API key. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

IndiaMART Scraper — B2B Suppliers & Contacts ✅ No Login

themineworks/indiamart-suppliers

Affordable IndiaMART scraper: B2B suppliers, phone, GST & leads. $2/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

India Data.gov Scraper — Government Open Datasets ✅ No API Key

themineworks/india-data-gov-scraper

Affordable India data.gov scraper: government open datasets. $1/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

Socrata Open Data Scraper — US Government Datasets ✅ No API Key

themineworks/socrata-open-data

Cheapest Socrata open-data scraper: any government dataset portal. $1/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works

Federal Register Scraper — US Rules & Regulations ✅

themineworks/federal-register-scraper

Affordable Federal Register scraper: US rules, notices & orders. $3/1,000 results, 25 free, pay-per-result, no subscription. Works in Claude, ChatGPT & any MCP-compatible AI agent. MCP server: the-mine-works-mcp.hatchable.site

The Mine Works