# CJK PDF to RAG Chunks – Traditional Chinese & Japanese OCR (`robertwang/cjk-pdf-rag-chunker`) Actor

Convert Traditional Chinese and Japanese PDFs (including scans) into heading-aware, sentence-safe chunks with page numbers for RAG and vector DBs. Parallel OCR, header/footer removal, optional Markdown tables, estimate mode. Pay per page.

- **URL**: https://apify.com/robertwang/cjk-pdf-rag-chunker.md
- **Developed by:** [Robert Wang](https://apify.com/robertwang) (community)
- **Categories:** AI, Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 ocr pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## CJK PDF to RAG Chunks (Traditional Chinese & Japanese)

Turn Traditional Chinese and Japanese PDFs, including scanned ones, into clean, heading-aware chunks that are ready for RAG, vector databases, and LLM pipelines.

### What it does

- **Text-layer PDFs**: extracts text with PyMuPDF and keeps the font-size information used for heading detection.
- **Scanned / image-only pages**: runs Tesseract OCR on several pages in parallel (150 dpi, grayscale).
  - Japanese documents automatically use `jpn+chi_tra`; Traditional Chinese uses `chi_tra` only.
  - Noise from charts and figures is filtered out.
- **CJK-aware chunking**:
  - splits at full-width sentence ends (`。！？；」`), never mid-sentence unless one sentence is longer than 2× the target size;
  - starts a new chunk at each page (an unfinished sentence carries over to the next page);
  - detects headings such as 第X章, 一、, （一）, 1.1, ■, and larger font sizes; numbers, years, and amounts are ignored as headings;
  - removes running headers and footers repeated across pages;
  - splits table-of-contents / dot-leader lines line by line (`type=toc`).
- **Optional table extraction**: tables become Markdown chunks (`type=table`).
- **Estimate mode**: counts pages and OCR pages and returns the price before you process anything.
- **Upload fallback**: some government sites block non-local IPs. Upload the PDF to a key-value store and pass its key.

### Input

```json
{
  "pdfUrls": ["https://example.gov.tw/report.pdf"],
  "uploadStore": "my-pdf-uploads",
  "uploadKeys": ["scan.pdf"],
  "mode": "extract",
  "chunkChars": 500,
  "maxPages": 200,
  "ocrLang": "",
  "detectTables": false,
  "pageBreak": true
}
```

| Field | Default | Notes |
|---|---|---|
| `pdfUrls` | – | Public PDF URLs |
| `uploadStore` / `uploadKeys` | – | Key-value store (read-only access) and record keys of uploaded PDFs |
| `mode` | `extract` | `estimate` = page count and price only, no output chunks |
| `chunkChars` | 500 | Target chunk length in characters |
| `maxPages` | 200 | Per-document cap |
| `ocrLang` | auto | Any Tesseract language string, e.g. `chi_tra`, `jpn`, `jpn+chi_tra` |
| `detectTables` | false | Markdown tables, billed as an add-on. About 0.1–0.4 s/page; switches itself off for the rest of that document (re-enabled for the next) if it averages >3 s/page. Long tables are split by rows with the header repeated; sparse chart-like grids are kept as text |
| `ocrDpi` | 150 | Higher = slower, sometimes more accurate |
| `forceOcr` | false | OCR every page even if a text layer exists |
| `ocrWorkers` | auto | Parallel OCR workers. Auto = the CPUs the run really has (cgroup quota / Apify memory: 1 CPU per 4 GB), max 8 |
| `verticalText` | `auto` | `auto` detects vertical (縦書き/直排) pages and uses `jpn_vert`/`chi_tra_vert`; `on` / `off` to force |
| `deskew` | true | Straighten pages tilted 0.8°–4.5° before OCR |
| `denoise` | true | Median filter, applied only to visibly noisy scans |
| `trustOcrLayer` | false | Scans often carry an invisible OCR layer with garbled CJK; by default such pages are re-OCRed. Set true to keep the existing layer |
| `maxRunSeconds` | – | Stop early and return what is done, marked `truncated` (reason `run time limit reached`). The platform run timeout is always respected too |

### Output (dataset item)

```json
{
  "url": "https://example.gov.tw/report.pdf",
  "status": "ok",
  "type": "text",
  "chunkIndex": 12,
  "text": "113 年各級政府經常門收入淨額 4 兆 1,596 億元……",
  "pageStart": 7,
  "pageEnd": 7,
  "headingPath": ["提要分析及專載", "二、公庫收支及政府債務", "1.中央政府債務"],
  "ocrLang": "chi_tra",
  "chars": 151
}
```

- `type` is `text`, `table`, or `toc`.
- Documents that fail produce `{"status": "failed", "error": "..."}`, and the run continues.
- Run statistics are saved to the key-value store record `STATS`.

### Pricing (pay per event)

| Event | Price |
|---|---|
| Actor start | $0.005 per run |
| Text-layer page | $3 per 1,000 pages |
| OCR page | $10 per 1,000 pages |
| Table extraction (only when `detectTables` is on) | $5 per 1,000 pages |

Example: a 300-page report with 40 scanned pages costs 260 × $0.003 + 40 × $0.010 + $0.005 = **$1.19**. Run `mode: "estimate"` first to see the exact price.

Platform usage is included. 4 GB of memory is recommended: OCR is much faster with more CPU.

### Limits

- OCR accuracy depends on scan quality. Expect occasional character errors (e.g. 入→人), and charts are only partly recovered.
- Vertical text works best on book-style, single-column pages.
- **Furigana / ruby (振り仮名):** the small reading hints beside kanji are not removed, so OCR output can contain stray kana inserted into words. Clean these up downstream if exact text matters.
- **Multi-column magazines and newspapers:** dense vertical or mixed-direction layouts with images (e.g. old magazines) are read in the wrong order and with many character errors (mean OCR confidence about 50). Treat their output as rough search text only.
- Scanned pages: headings come from numbering (第一章 / 一、 / （一） / 1.1 …); size-based headings are used only on high-confidence horizontal OCR.
- Download failures are retried 3 times with backoff (429/5xx/timeouts). Sites that block data-center IPs: upload the PDF to a key-value store and use `uploadStore` + `uploadKeys` (no paid proxy is used).
- OCR on the default 4 GB memory (1 CPU) runs about 3–5 s per scanned page.
- Complex multi-column layouts may interleave columns.
- Simplified Chinese and Korean are not enabled by default. Set `ocrLang`, e.g. `chi_sim` or `kor`, if those packs are installed.
- Password-protected PDFs are not supported.

# Actor input Schema

## `pdfUrls` (type: `array`):

Public PDF URLs

## `mode` (type: `string`):

estimate = count pages / OCR pages only

## `chunkChars` (type: `integer`):

Approx characters per chunk

## `maxPages` (type: `integer`):

Hard cap

## `forceOcr` (type: `boolean`):

Testing: OCR even if text layer exists

## `ocrLang` (type: `string`):

Tesseract langs. Empty = auto (jpn+chi_tra for Japanese docs, else chi_tra+jpn)

## `ocrDpi` (type: `integer`):

Render DPI for OCR

## `ocrWorkers` (type: `integer`):

Default: CPU count (min 2)

## `detectTables` (type: `boolean`):

Off by default (slow, ~7 s/page). Emit tables as markdown chunks with type=table

## `pageBreak` (type: `boolean`):

Flush at page end (unfinished sentence carried over)

## `uploadStore` (type: `string`):

Key-value store holding uploaded PDFs (use when the source site blocks download).

## `uploadKeys` (type: `array`):

Record keys in the upload store (or this run's default store if no store chosen).

## `verticalText` (type: `string`):

Scanned pages: detect vertical (top-to-bottom, right-to-left) text and OCR with jpn_vert / chi_tra_vert.

## `deskew` (type: `boolean`):

Straighten scanned pages tilted 0.8–4.5°.

## `denoise` (type: `boolean`):

Median filter, applied only to pages with heavy speckle noise.

## `trustOcrLayer` (type: `boolean`):

Scanned PDFs often carry a hidden OCR text layer of poor quality for CJK. By default such pages are re-OCRed; enable to use the existing layer (cheaper, billed as text pages).

## `maxRunSeconds` (type: `integer`):

Stop early (marking results 'truncated') before this many seconds. The platform run timeout is always respected too. / 超過秒數前提早停止並標記 truncated；平台逾時也會自動遵守。

## Actor input object example

```json
{
  "pdfUrls": [
    "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
  ],
  "mode": "extract",
  "chunkChars": 500,
  "maxPages": 200,
  "forceOcr": false,
  "ocrDpi": 150,
  "detectTables": false,
  "pageBreak": true,
  "verticalText": "auto",
  "deskew": true,
  "denoise": true,
  "trustOcrLayer": false
}
```

# Actor output Schema

## `chunks` (type: `string`):

One dataset item per chunk (plus status rows for estimate/failed/truncated documents).

## `stats` (type: `string`):

Page counts, OCR pages and timing for the run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "pdfUrls": [
        "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("robertwang/cjk-pdf-rag-chunker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "pdfUrls": ["https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"] }

# Run the Actor and wait for it to finish
run = client.actor("robertwang/cjk-pdf-rag-chunker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "pdfUrls": [
    "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
  ]
}' |
apify call robertwang/cjk-pdf-rag-chunker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,robertwang/cjk-pdf-rag-chunker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CzLEmWpA1YjYKnHgz/builds/WHPqLogJGF0fTp86E/openapi.json