Knowledge Delta | RAG Knowledge Base Change Detection

Knowledge Intelligence Engine — Website to Markdown for RAG

ryanclinton/website-content-to-markdown

Turn any website, documentation site or help centre into a retrieval-ready knowledge corpus for RAG and AI search. Clean Markdown plus chunks, change detection, deduplication, retrieval scoring, version awareness and a full corpus audit, in one run.

Rag Knowledge Graph Builder

cspnair/rag-knowledge-graph-builder

Transform websites into RAG-ready datasets. Crawls pages, chunks content into semantic segments (500-1000 tokens), and generates hypothetical questions for each chunk. No API key needed with native mode. Output: pre-indexed JSON optimized for AI retrieval with 3x better accuracy than raw text.

RAG-Ready Web Scraper & Smart Chunker for AI Knowledge Bases

adinfosys-labs/rag-ready-web-scraper-smart-chunker-for-ai-knowledge-bases

RAG-ready web scraper that collects, cleans, deduplicates, filters, and chunks web content into structured datasets for AI pipelines. Generates high-quality knowledge-base data optimized for LLMs, embeddings, and vector databases

Monitor documentation, help centers and policy pages. Get readable changes, stable document IDs, chunk upserts and confirmed deletion events for AI knowledge bases. No LLM API key required.

⚡ Web to LLM Clean Markdown & RAG Crawler

unrivaled_fortress/llm-markdown-clean-web-crawler

Convert any webpage into clean, LLM-ready markdown. Strips ads, navbars, and cookie banners to optimize AI context tokens for ChatGPT, Claude, and RAG.

Create deduplicated, metadata-rich chunks for RAG ingestion.

Monitor public web pages, persist a baseline, and return compact change events for meaningful content changes.

PDF Page Text Extractor – No Charge for Blank Pages

rowfeed/pdf-page-text-extractor

Extract text from any PDF URL, one row per page. Pages with no extractable text (scanned PDFs) are never charged — the main differentiator. pypdf parsing, no OCR. Clean JSON for RAG pipelines, researchers and automation.

Compare XML sitemap snapshots for new URLs, metadata edits and entries absent from the latest document. Keep persistent history, return the current inventory and deduplicatable change IDs, and distinguish URL sets from sitemap indexes.

Website to AI / RAG Data Extractor

scrapyx/website-rag-extractor

Turns any website into a clean, embedding-ready corpus. Strips navigation, footers and cookie banners, converts the real content to markdown, and splits it into overlapping chunks that each carry their own URL, title and metadata. Crawl by URL list, sitemap or link graph.