CJK PDF to RAG Chunks (Traditional Chinese & Japanese)

Turn Traditional Chinese and Japanese PDFs, including scanned ones, into clean, heading-aware chunks that are ready for RAG, vector databases, and LLM pipelines.

What it does

• Text-layer PDFs: extracts text with PyMuPDF and keeps the font-size information used for heading detection.
• Scanned / image-only pages: runs Tesseract OCR on several pages in parallel (150 dpi, grayscale).
  • Japanese documents automatically use jpn+chi_tra; Traditional Chinese uses chi_tra only.
  • Noise from charts and figures is filtered out.
• CJK-aware chunking:
  • splits at full-width sentence ends (。！？；」), never mid-sentence unless one sentence is longer than 2× the target size;
  • starts a new chunk at each page (an unfinished sentence carries over to the next page);
  • detects headings such as 第X章, 一、, （一）, 1.1, ■, and larger font sizes; numbers, years, and amounts are ignored as headings;
  • removes running headers and footers repeated across pages;
  • splits table-of-contents / dot-leader lines line by line (type=toc).
• Optional table extraction: tables become Markdown chunks (type=table).
• Estimate mode: counts pages and OCR pages and returns the price before you process anything.
• Upload fallback: some government sites block non-local IPs. Upload the PDF to a key-value store and pass its key.

Input

{
  "pdfUrls": ["https://example.gov.tw/report.pdf"],
  "uploadStore": "my-pdf-uploads",
  "uploadKeys": ["scan.pdf"],
  "mode": "extract",
  "chunkChars": 500,
  "maxPages": 200,
  "ocrLang": "",
  "detectTables": false,
  "pageBreak": true
}

Field	Default	Notes
pdfUrls	–	Public PDF URLs
uploadStore / uploadKeys	–	Key-value store (read-only access) and record keys of uploaded PDFs
mode	extract	estimate = page count and price only, no output chunks
chunkChars	500	Target chunk length in characters
maxPages	200	Per-document cap
ocrLang	auto	Any Tesseract language string, e.g. chi_tra, jpn, jpn+chi_tra
detectTables	false	Markdown tables, billed as an add-on. About 0.1–0.4 s/page; switches itself off for the rest of that document (re-enabled for the next) if it averages >3 s/page. Long tables are split by rows with the header repeated; sparse chart-like grids are kept as text
ocrDpi	150	Higher = slower, sometimes more accurate
forceOcr	false	OCR every page even if a text layer exists
ocrWorkers	auto	Parallel OCR workers. Auto = the CPUs the run really has (cgroup quota / Apify memory: 1 CPU per 4 GB), max 8
verticalText	auto	auto detects vertical (縦書き/直排) pages and uses jpn_vert/chi_tra_vert; on / off to force
deskew	true	Straighten pages tilted 0.8°–4.5° before OCR
denoise	true	Median filter, applied only to visibly noisy scans
trustOcrLayer	false	Scans often carry an invisible OCR layer with garbled CJK; by default such pages are re-OCRed. Set true to keep the existing layer
maxRunSeconds	–	Stop early and return what is done, marked truncated (reason run time limit reached). The platform run timeout is always respected too

Output (dataset item)

{
  "url": "https://example.gov.tw/report.pdf",
  "status": "ok",
  "type": "text",
  "chunkIndex": 12,
  "text": "113 年各級政府經常門收入淨額 4 兆 1,596 億元……",
  "pageStart": 7,
  "pageEnd": 7,
  "headingPath": ["提要分析及專載", "二、公庫收支及政府債務", "1.中央政府債務"],
  "ocrLang": "chi_tra",
  "chars": 151
}

• type is text, table, or toc.
• Documents that fail produce {"status": "failed", "error": "..."}, and the run continues.
• Run statistics are saved to the key-value store record STATS.

Pricing (pay per event)

Event	Price
Actor start	$0.005 per run
Text-layer page	$3 per 1,000 pages
OCR page	$10 per 1,000 pages
Table extraction (only when detectTables is on)	$5 per 1,000 pages

Example: a 300-page report with 40 scanned pages costs 260 × $0.003 + 40 × $0.010 + $0.005 = $1.19. Run mode: "estimate" first to see the exact price.

Platform usage is included. 4 GB of memory is recommended: OCR is much faster with more CPU.

Limits

• OCR accuracy depends on scan quality. Expect occasional character errors (e.g. 入→人), and charts are only partly recovered.
• Vertical text works best on book-style, single-column pages.
• Furigana / ruby (振り仮名): the small reading hints beside kanji are not removed, so OCR output can contain stray kana inserted into words. Clean these up downstream if exact text matters.
• Multi-column magazines and newspapers: dense vertical or mixed-direction layouts with images (e.g. old magazines) are read in the wrong order and with many character errors (mean OCR confidence about 50). Treat their output as rough search text only.
• Scanned pages: headings come from numbering (第一章 / 一、 / （一） / 1.1 …); size-based headings are used only on high-confidence horizontal OCR.
• Download failures are retried 3 times with backoff (429/5xx/timeouts). Sites that block data-center IPs: upload the PDF to a key-value store and use uploadStore + uploadKeys (no paid proxy is used).
• OCR on the default 4 GB memory (1 CPU) runs about 3–5 s per scanned page.
• Complex multi-column layouts may interleave columns.
• Simplified Chinese and Korean are not enabled by default. Set ocrLang, e.g. chi_sim or kor, if those packs are installed.
• Password-protected PDFs are not supported.