Pricing

$15.00 / 1,000 rag chunks

Techionik Website RAG MCP

Prompt-first MCP-ready Apify tool for AI agents to crawl public websites and return clean RAG chunks with URLs, titles, markdown, and token estimates.

Pricing

$15.00 / 1,000 rag chunks

Rating

0.0

(0)

Developer

Techionik

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

What it does

Accepts a natural-language MCP prompt for agent-friendly use.
Also accepts structured JSON for batch/API workflows.
Returns stable dataset records with status, warnings, source URLs, and useful business fields.
Keeps the workflow lightweight by using public pages and clear failure records.

Input

Use the prompt field when calling this as an MCP-style tool:

{
    "prompt": "Create a RAG dataset from https://example.com/docs. Crawl up to 8 same-domain pages, chunk around 1600 characters, include markdown, titles, source URLs, and token estimates."
}

Structured fields remain available for API users who want explicit batch control.

Local testing

Run the unit tests and project validator from this actor folder:

npm test
npm run validate

Apify deployment

Before deployment, confirm the active Apify account is Techionik:

apify info
apify push

Marketplace publication is intentionally skipped until final business approval.

Output

Results are written to the default Apify dataset. Each record includes a status field, extracted signals, warnings when something could not be fetched, and the original source input for traceability.

Pricing

Recommended pricing: USD 1.50 per 1,000 chunks or USD 0.60 per 1,000 pages during Store billing setup. The output is immediately useful for RAG ingestion, documentation search, and AI support workflows.

Limitations

Works on public pages only.
Does not bypass logins, captchas, paywalls, or site restrictions.
Some JavaScript-heavy sites may return partial data with a warning.
Final Apify Store billing still needs to be selected during marketplace publication.

MCP Website RAG Ingestion Tool

glowing_glove/mcp-website-rag-ingestion-tool

MCP-ready website ingestion for AI agents using a prompt-first interface that turns public webpages into source-linked RAG chunks with markdown and token estimates.

Ushba Khan

RAG Website Crawler - Markdown Chunks for LLMs & MCP

themineworks/rag-crawler

Crawl any website into clean, pre-chunked Markdown with per-chunk token counts for RAG pipelines, vector DBs (Pinecone, Qdrant) and LLM context. MCP-native for Claude & ChatGPT. SPA support via Playwright. Pay only for pages that crawl. A Firecrawl alternative.

The Mine Works

Actor-to-MCP Bridge — Expose Any Apify Actor as an AI Tool

busy_donkey/actor-mcp-bridge

Wraps any Apify actor as an MCP (Model Context Protocol) server. Connect Claude, GPT, or any AI agent to run Apify scrapers as native tools.

busy_donkey

AI-Ready Content Extractor — Structured Web Data for LLM & MCP

yuchiaoniu/ai-content-extractor

Extract structured JSON from any URL for LLM, RAG, and MCP integration. Outputs title, sections, contact info, links, structured data, and clean plain text.

Niu Yuchiao

AI Web Extract — Structured Data from Any URL

logiover/ai-web-extract

Give a URL, get clean structured JSON — no LLM, no API key. Keyless Firecrawl Extract alternative that pulls schema.org JSON-LD, OpenGraph/meta, microdata, tables, prices, dates and contacts from any page. Built for AI agents, RAG and MCP.

Logiover

Website to Markdown Scraper - HTML to MD for RAG API

pink_comic/website-content-to-markdown

Convert web pages and HTML to clean Markdown for RAG, LLM training, AI knowledge bases, and content migration. Strips nav, ads, scripts, and styling while preserving structure. Bulk output includes word/link/image counts.

Ava Torres

Web Scraper 🚀

datascoutapi/web-scraper

Web Scraper Pro extracts clean structured data for LLMs/RAG. Browser-based, 10x faster with anti-detection bypassing Cloudflare/CAPTCHA & proxy rotation. Bulk/recursive crawl 50k URLs at 500 pages/min. JSON/CSV/API, free tier.

halam

🧠 RAG Web Browser — Web Content for AI & LLMs

nexgendata/rag-web-browser

Web browser for RAG pipelines and AI agents. Search Google, scrape top results, return clean Markdown. Feed your LLM with real-time web data. Works with Claude, GPT, LangChain, CrewAI. No API key needed.

NexGenData

RSS Feed Reader (with MCP)

fetch_cat/rss-feed-reader

Read RSS and Atom feeds, normalize feed items, and export clean dataset rows for monitoring, alerts, AI, MCP, and integrations.

Hanna Nosova

Github Tech Intelligence

credible_sandal/github-tech-intelligence

GitHub Trending highlights what’s popular today. This surfaces what’s about to take off tomorrow—and shows which repos are healthy, safe to adopt, and truly gaining traction.