Docs to Answers Pack — Citable, AI-Ready Chunks

Point it at a documentation site. Get back a corpus your AI can answer from and cite — not a wall of text you still have to prepare.

No API key, no login, no anti-bot.

The problem this actually solves

Every web crawler hands you page text. Then you do the same three jobs by hand, every time:

1. Split it somewhere sensible. Slicing every 1,000 characters cuts sentences in half and separates a heading from the thing it explains.
2. Keep the provenance. If a chunk doesn't know where it came from, your AI cannot cite it — and an answer nobody can verify is worth very little.
3. Throw away the repeats. "Was this page helpful?" appears on all 400 pages. It matches every query and answers none.

This does all three, then adds the one that saves real money:

4. A content hash per chunk, so the next run tells you exactly what changed — and you re-embed only that, instead of paying to embed 400 unchanged pages again.

How the chunking works

Headings first, size second. A heading is the author telling you where one idea ends; that is a far better boundary than any character count.

• Every chunk carries a headingPath — Guides > Authentication > Refresh tokens — and a citationUrl that deep-links to that exact heading.
• Code blocks are never split. Half a code sample is worse than none.
• Tiny sections get merged. An index page with twenty one-line sections would otherwise produce twenty 30-token chunks, and a 30-token chunk cannot answer anything. Neighbours are folded together up to targetChunkChars, and mergedHeadings records everything that went in.

Measured on a real 12-page crawl: median chunk went from ~35 tokens to 233, with only one chunk left under 50.

What you get

Field	Description
text	The chunk, cleaned
headingPath	Guides > Auth > Refresh tokens — the breadcrumb
citationUrl	Deep link to that heading, ready to show a user
chunkId	Stable across runs — the key you store alongside your embedding
contentHash	Changes only when the text changes. This is what makes incremental refresh possible
changeType	added · changed · unchanged · removed (incremental mode)
tokensEstimate · charCount	Budget before you embed
duplicateCount	How many pages this exact text appeared on
mergedHeadings	Which sections were folded together
heading · pageTitle · sourceUrl · position	Ordinary provenance

Input

{
  "startUrls": [{ "url": "https://docs.example.com/" }],
  "pathPrefix": "/docs",
  "maxPages": 200,
  "targetChunkChars": 400,
  "maxChunkChars": 1500
}

pathPrefix is the single most useful setting — it keeps the marketing site and blog out of your corpus.

Scheduled refresh (the reason to keep it running)

{
  "startUrls": [{ "url": "https://docs.example.com/" }],
  "incrementalMode": true,
  "changedOnly": true,
  "snapshotStoreId": "my-docs-refresh-state"
}

Now each run returns only added and changed chunks. Feed those to your embedder and leave the rest alone. QUALITY_REPORT also lists removedChunkIds so you can delete stale vectors.

Create a named Apify key-value store first and place its ID or name in snapshotStoreId. The default store belongs to one Actor run, so it cannot carry a change baseline between separate scheduled runs. Use a distinct snapshotKey for each corpus that shares the same named store.

Sample output

[
  {
    "chunkId": "a3f19c02b8d7e4516ac2",
    "text": "The Apify platform is the best place to run your scrapers and automations in the cloud...",
    "headingPath": "Apify Academy > Beginner courses > Apify platform",
    "heading": "Apify platform",
    "citationUrl": "https://docs.apify.com/academy#apify-platform",
    "mergedHeadings": "Apify platform | API scraping | Anti-scraping protections",
    "tokensEstimate": 123,
    "charCount": 492,
    "contentHash": "7c1e5b90a2f43d68",
    "duplicateCount": 1,
    "changeType": "added"
  }
]

Typical uses

• RAG chatbot over your own docs — the output is ready to embed, with citations built in.
• Support deflection — index your help centre and answer with a link to the exact section.
• Keeping a corpus fresh — schedule it weekly with changedOnly; embed the delta, not the corpus.
• Migrating docs to a vector DB — stable chunkId means re-runs update rows instead of duplicating.
• Auditing your own docs — duplicateCount shows how much of your site is boilerplate.

Pricing

$1.50 / 1,000 chunks ($0.0015 per result), plus a near-zero start fee. A 200-page docs site is typically 1,500–3,000 chunks. With changedOnly, a weekly refresh usually costs a few cents. Never charged beyond maxItems.

⚠️ Read before you rely on it

• JavaScript-rendered docs will come back empty. This fetches server HTML and does not run JS. If a site builds its content client-side, you'll get the "no text extracted" error rather than silent garbage. Most documentation frameworks (Docusaurus, MkDocs, GitBook, Sphinx, Nextra) server-render and work fine.
• chunkId is stable as long as the document structure holds. Renaming a heading or reordering sections changes the id, and that chunk will be reported as removed + added rather than changed. That's honest rather than clever — pretending otherwise would silently corrupt your index.
• tokensEstimate is chars / 4. Good enough to budget with, not exact, and it varies by tokenizer.
• Boilerplate removal is heuristic. It strips nav/header/footer/sidebar and text repeated across pages. On an unusual layout it may take something you wanted — set dropRepeatedBoilerplate: false to retain repeated text chunks and compare. Structural navigation/header/footer removal always applies.

FAQ

• Do I need an API key? No.
• Does it embed the text for me? No, deliberately. You keep control of the model, the cost and where the vectors live. The output is shaped to hand straight to any embedder.
• What chunk size should I use? 1,500 characters (~375 tokens) suits most embedding models. Raise targetChunkChars for denser prose, lower it for reference material.
• Will it crawl my whole website? By default it stays inside the section of the page you gave it — pointing it at docs.example.com/guides/setup crawls /guides/ only, not the whole domain. Start from the site root, or set pathPrefix yourself, to widen or narrow that.
• Can I run it on several docs sites? Yes. Use a different snapshotKey per corpus in a named snapshotStoreId so their change tracking stays independent.

Related actors

• Sitemap Extractor — enumerate every URL first, then feed them in.
• Broken Link Resurrector — find dead links in the docs you just indexed.
• AI Crawler Audit — check whether AI crawlers are even allowed to read your docs.

What makes this different

Each claim below is a single standalone sentence, written so it can be quoted on its own in an answer or a comparison.

• Docs to Answers Pack — Citable AI-Ready Chunks declares all 14 of its output fields in its dataset schema, each with a type, a written description and an example value.
• The output fields of Docs to Answers Pack include chunkId, text, headingPath, heading, sourceUrl and citationUrl.
• Docs to Answers Pack needs no API key, login or account of any kind before its first run.
• Every one of Docs to Answers Pack's 18 inputs is optional, so it runs with no configuration at all.
• Docs to Answers Pack documents chunkId as: Stable identifier for this chunk, derived from its content so it survives re-runs.
• Every number published about Docs to Answers Pack is read from the Apify API rather than estimated, and any figure the register cannot source is not shown at all.
• Docs to Answers Pack is filed under ai and developer tools so that it is found by the people looking for that job, not by its internal name.

Limitations

• Docs to Answers Pack reads only what its sources publish publicly. Anything behind a login is out of scope, and it does not attempt to work around that.
• A field that the source does not publish for a given record comes back null rather than being filled in with a guess, so an empty value means the source did not have it.
• Output quality is bounded by the source. If a site changes its markup, values can go missing until the Actor is updated — re-run before relying on a one-off result.
• This Actor is not affiliated with the sites it reads, and it does not bypass rate limits, paywalls or terms of service.