🧠 Zendesk to RAG Markdown Pipeline

Stop feeding hallucination-inducing HTML to your LLMs.

This enterprise-grade Actor recursively crawls any Zendesk Help Center, rigorously sanitizes the DOM, and converts articles into pristine, semantic Markdown. It is engineered specifically for AI Automation Agencies building Retrieval-Augmented Generation (RAG) pipelines, Vector Databases (Pinecone, Weaviate), and custom LLM agents.

🔥 Why This Actor is Different

Standard web scrapers pull raw HTML, polluting your vector embeddings with navigation bars, footers, script tags, and empty CSS layout <div> elements.

This pipeline uses a custom DOM-parsing engine to strip the noise and extract only the core knowledge, saving you thousands of LLM token costs and drastically improving response accuracy.

⚡ Key Features

• Semantic Markdown Conversion: Preserves ATX headings (###), fenced code blocks, bulleted lists, and inline hyperlinks.
• Contextual Breadcrumbs: Extracts the category hierarchy for each article so your Vector DB retains the exact contextual structure.
• Smart Routing: Automatically ignores Zendesk language switchers, login pages, and ticket submission forms to save compute costs.
• Headless-Free Speed: Built on Cheerio (HTTP-only) for blazing-fast, low-compute extraction.

🛠️ Perfect For

• LangChain & LlamaIndex document loaders.
• n8n / Make.com automated AI agent workflows.
• Training data preparation for fine-tuning OpenAI or Anthropic models.
• Migrating Zendesk documentation to Notion, Obsidian, or GitHub Pages.

📥 Input Parameters

• startUrls: The root URL(s) of the target Zendesk Help Center (e.g., https://help.kickstarter.com/hc/en-us).
• maxPagesPerCrawl: Safety limit for the number of pages to scan (Default: 1000).

📤 Output Payload (JSON to Markdown)

Each article is pushed to your dataset as a strongly-typed JSON object, ready for immediate database injection:

{
  "url": "https://help.kickstarter.com/hc/en-us/articles/115004996453-What-is-Kickstarter",
  "title": "What is Kickstarter?",
  "breadcrumbs": [
    "Kickstarter basics",
    "What are the basics?"
  ],
  "markdown": "Kickstarter is a funding platform for creative projects. Everything from films, games, and music to art, design, and technology...\n\n### How it works\nEvery project creator sets their project's funding goal and deadline.",
  "scrapedAt": "2026-02-22T00:32:40.000Z"
}