# HelpDesk-to-RAG Markdown Scraper (`neighborly_coach/helpdesk-to-rag-markdown-scraper`) Actor

Extract clean, RAG-ready Markdown from any knowledge base for LLM training and AI chatbots.

- **URL**: https://apify.com/neighborly\_coach/helpdesk-to-rag-markdown-scraper.md
- **Developed by:** [Alona Ostrenska](https://apify.com/neighborly_coach) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🤖 HelpDesk-to-RAG Markdown Scraper

Transform messy knowledge bases and help centers into perfectly clean, structured Markdown—ready for instant ingestion by Large Language Models (LLMs) and RAG (Retrieval-Augmented Generation) pipelines.

### 🚀 Why Use This Scraper?

When building custom AI agents or AI customer support bots, feeding them raw HTML confuses the models with navigation bars, tracking scripts, and footer links. This wastes your API tokens and degrades the AI's answers.

This Actor solves that problem. It uses **Mozilla's Readability Engine** to isolate the core article content and **Turndown** to convert it into pristine Markdown.

- **Lower Token Costs:** Stripping out heavy HTML tags saves massive amounts of tokens when passing context to OpenAI, Anthropic, or local LLMs.
- **Better Vector Search:** Clean Markdown headers (`#`, `##`) allow semantic chunking tools (like LangChain or LlamaIndex) to split your documents intelligently.
- **Zero Maintenance:** It automatically handles internal link discovery, proxy rotation, and pagination across domains.

### ⚙️ Core Features

- **Deep Crawling:** Provide a single starting URL (e.g., `https://docs.apify.com`), and the Actor will automatically find and queue all other articles on that same domain.
- **Smart Content Extraction:** Automatically identifies and extracts only the main article, ignoring sidebars, ads, and footers.
- **Anti-Blocking Built-In:** Fully integrated with Apify's Proxy network to bypass rate limits and IP bans without any configuration required on your end.

### 📝 Input Configuration

This Actor is designed to be simple to use. Here are the available inputs:

| Field | Type | Description | Default |
| :--- | :--- | :--- | :--- |
| **Start URLs** | `Array` | The URLs of the help centers or specific articles you want to scrape. | `[{"url": "https://docs.apify.com/"}]` |
| **Max Pages** | `Integer` | The absolute maximum number of pages to crawl before the Actor stops. | `50` |
| **Remove Boilerplate** | `Boolean` | Strips navbars, ads, and footers before converting to Markdown. Highly recommended for AI use cases. | `true` |

### 💰 Cost and Pricing

This Actor operates on a highly efficient Pay-per-event (PPE) model. You only pay for what you actually get!
Price: $4.00 per 1,000 results ($0.004 per successfully extracted Markdown article).
Platform Costs: Free / Included! The Apify platform computing costs are fully baked into the per-article price.
You are never charged for failed page loads, redirects, or blocked requests.

### 🔗 Seamless Integrations

Because this Actor outputs clean JSON and Markdown, it can be easily connected to your favorite AI tools using the Apify API or Webhooks:
Vector Databases: Pinecone, Milvus, Weaviate, Qdrant
AI Frameworks: LangChain, LlamaIndex, Flowise
Automation: Zapier, Make.com
Built with ❤️ for the AI developer community.

### 📊 Output Data Format

The Actor stores its results in the Apify Dataset. You can instantly download your data in JSON, CSV, XML, or Excel.

Here is an example of a single extracted article in JSON format:

````json
{
  "url": "[https://docs.apify.com/academy/getting-started](https://docs.apify.com/academy/getting-started)",
  "title": "Getting Started | Apify Documentation",
  "markdown": "# Getting Started\n\nWelcome to the Apify platform! In this guide, you will learn how to build your first web scraper...\n\n## Prerequisites\n* A free Apify account.\n* Basic knowledge of JavaScript.",
  "scrapedAt": "2026-07-28T12:00:00.000Z"
}

# Actor input Schema

## `startUrls` (type: `array`):

URLs of the help centers to scrape.
## `maxPagesPerCrawl` (type: `integer`):

Maximum number of articles to extract.
## `removeBoilerplate` (type: `boolean`):

Automatically strip non-article content before converting to Markdown.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://docs.apify.com/"
    }
  ],
  "maxPagesPerCrawl": 50,
  "removeBoilerplate": true
}
````

# Actor output Schema

## `results` (type: `string`):

The dataset containing all scraped knowledge base articles.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://docs.apify.com/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("neighborly_coach/helpdesk-to-rag-markdown-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://docs.apify.com/" }] }

# Run the Actor and wait for it to finish
run = client.actor("neighborly_coach/helpdesk-to-rag-markdown-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://docs.apify.com/"
    }
  ]
}' |
apify call neighborly_coach/helpdesk-to-rag-markdown-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=neighborly_coach/helpdesk-to-rag-markdown-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/BkyzHoFz1hysNhdW4/builds/8fJMXZhvS7cJ4cazi/openapi.json