Paste a URL and get a chatbot-ready knowledge base in one run: clean Markdown, embedding-ready chunks, FAQs, llms.txt, products and a site profile.

• RAG-ready chunks: heading-aware, with context headers, stable IDs and token counts. Never split inside links, tables or code.
• Less noise: removes menus, cookie banners, chat widgets, repeated blocks and near-duplicate pages.
• Auto rendering: fast HTTP first, a browser only for JavaScript or blocked pages.
• Incremental mode: scheduled runs output only new and changed pages and list removed ones. Unchanged pages are free.
• FAQs & products extracted and de-duplicated. Optional PDFs.
• Optional AI (your Anthropic key): summaries, key facts and a chatbot system prompt.

• Dataset: one row per page, or per chunk for vector-DB integrations (Pinecone, Qdrant…)
• KNOWLEDGE_BASE.md: upload to a Custom GPT or Claude Project
• chunks.jsonl, faqs.jsonl, products.jsonl, llms.txt, llms-full.txt
• RUN_REPORT: what was skipped and why, with hints

Event	Per 1,000
Page saved	$0.80
+ browser rendering	+$1.20
PDF	$2.00
AI enrichment	$1.00

Thin, duplicate, blocked and unchanged pages are free. Runs stop cleanly at your cost limit.

1. Add a homepage, docs or help-center URL.
2. Click Start (defaults suit most sites).
3. Download KNOWLEDGE_BASE.md or use the chunks dataset.

Only crawl sites you may crawl. robots.txt is respected by default.