Website Embeddings for RAG - Vector DB, Pinecone, Qdrant

Website Classifier & Text Classifier - Industry & Sentiment

tidytools/ai-page-classifier

Classify URLs, domains or texts with AI: industry with B2B/B2C, page type, topic, IAB category, sentiment, content moderation or your own labels. Each row has a confidence and a reason. $4 per 1,000 items (Standard). Unreadable pages are free.

Website & Page to Markdown for RAG and LLMs

humble-echidna/page-to-markdown

Turn web pages or whole sites into clean Markdown for RAG and LLMs: main content only, with title, links and word count. Whole-site mode reads the sitemap first, can return only changed pages, and splits pages into heading-based chunks. Respects robots.txt and AI opt-outs. No browser.

Knowledge Intelligence Engine — Website to Markdown for RAG

ryanclinton/website-content-to-markdown

Turn any website, documentation site or help centre into a retrieval-ready knowledge corpus for RAG and AI search. Clean Markdown plus chunks, change detection, deduplication, retrieval scoring, version awareness and a full corpus audit, in one run.

Markdown RAG Chunker: Document Text Splitter

codepoetry/markdown-rag-chunker

RAG chunking for PDFs, Word, PowerPoint, Excel, EPUB and web pages. A header-aware text splitter keeps each chunk inside one section, caps it at 512 tokens by default, and adds its header path, source, stable ID and exact token count. Ready to embed into Pinecone, Qdrant or pgvector.

AI Website Content Extractor

scrapeai/ai-website-content-extractor

Crawl website pages, strip noise, and convert the main content to clean Markdown for RAG/LLM training.

Google Trends Scraper - Interest, Regions, Trending Now

tidytools/google-trends-scraper

Compare Google Trends keywords by country and time range. Input {"searchTerms":["chatgpt, claude"],"geo":"US"} returns interest over time, by region, related and rising queries, plus Trending Now with volume. $2 per 1,000 term reports. Rate-limited or failed searches are free.

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

AST Website Content Crawler for RAG

lukas459/ai-web-to-markdown-crawler-llm-rag-optimized

Advanced web crawler extracting token-chunked Markdown for LLMs & RAG. Features dual-engine hybrid parsing (IBM Docling & Trafilatura) to preserve tables and layouts. Delivers clean text split by native tiktoken counts with rich metadata (headings, levels, tokens) for instant vector search.

Download any PDF and chunk into semantically coherent segments ready for embedding/RAG. Configurable chunk size + overlap. Returns one row per chunk with page, char count, token estimate. Feed directly into OpenAI text-embedding-3 / Voyage / Cohere. $0.005 per PDF + $0.0002 per chunk.

Strip boilerplate, cookie banners, and nav clutter to extract high-density Markdown chunks, pricing tables, and typed JSON schema entities for RAG & MCP.