# Changelog of Website to RAG Chunks: Markdown, Tokens & Incremental Crawl (`nacenempresas/rag-website-chunks`) Actor

- **URL**: https://apify.com/nacenempresas/rag-website-chunks/changelog.md
- **Full Actor documentation**: https://apify.com/nacenempresas/rag-website-chunks.md

## Changelog

### 1.0 – 2026-11

- First public version.
- Heading-aware chunking with a hard token limit (section → paragraph → sentence).
- Exact token counts with tiktoken (`cl100k_base`, `o200k_base`).
- HTTP-first crawling with automatic headless-browser fallback for empty pages.
- PDF, DOCX, TXT, Markdown and llms.txt support.
- Incremental mode with ETag, Last-Modified, sitemap `lastmod` and content fingerprints.
- Boilerplate, duplicate-chunk and near-duplicate-page removal.
- `llms.txt`, full-site Markdown and run summary saved to the key-value store.
- Pay-per-page pricing; errors, duplicates and unchanged pages are never charged.