# Split Japanese documents into RAG chunks with overlap

**Use case:** 

Cleans Japanese documents and splits them into chunks for retrieval-augmented generation (RAG). Chunks end at sentence boundaries (。！？ and line breaks) instead of in the middle of a word, with an overlap you choose, and keep the metadata fields you name (for example url and title). Each chunk row has its position in the document, ready to embed and load into a vector database.

## Input

```json
{
  "texts": [
    "第1章　サービスの概要\n本サービスは、社内の文書を検索しやすくするための仕組みです。文書を取り込むと、自動で段落に分けて索引を作ります。\n第2章　使い方\n管理画面から文書をアップロードしてください。ＰＤＦ、Ｗｏｒｄ、テキストに対応しています。取り込みには数分かかることがあります。検索欄に質問を入力すると、関係する段落が表示されます。\n第3章　よくある質問\n取り込める文書の数に上限はありますか？　現在は1組織あたり1万件までです。上限を増やしたい場合はお問い合わせください。"
  ],
  "textFields": [
    "text"
  ],
  "metadataFields": [
    "url",
    "title"
  ],
  "splitIntoChunks": true,
  "maxChunkChars": 200,
  "chunkOverlapChars": 40,
  "unicodeNormalization": "nfc",
  "fullwidthAscii": "alphanumeric",
  "halfwidthKatakana": true,
  "composeVoicedMarks": true,
  "presentationForms": true,
  "unifyDashes": true,
  "removeInvisibleCharacters": true,
  "normalizeWhitespace": true,
  "spacesBetweenJapanese": "letterSpaced",
  "removeSpacesBetweenJapaneseAndLatin": false,
  "joinBrokenLines": true,
  "removePageNumbersAndHeaders": false,
  "punctuationStyle": "keep"
}
```

## Output

```json
{
  "documentIndex": {
    "label": "Document index",
    "format": "integer"
  },
  "chunkIndex": {
    "label": "Chunk index",
    "format": "integer"
  },
  "charCount": {
    "label": "Characters",
    "format": "integer"
  },
  "text": {
    "label": "Text",
    "format": "string"
  },
  "metadata": {
    "label": "Metadata",
    "format": "object"
  }
}
```

## How it works

### 1. Sign up on Apify or log in

The Japanese Text Normalizer & RAG Chunker Actor is ready to run with the inputs already set for "Split Japanese documents into RAG chunks with overlap". [Sign up or log in](https://console.apify.com/create-task-from-example/cgidIZDwkgpiq62Vr) and you'll land in Apify Console with a copy of this exact task.

### 2. Start the "Split Japanese documents into RAG chunks with overlap" task

The Japanese Text Normalizer & RAG Chunker Actor will start running based on the input automatically. You can also change all the input fields, like texts, text fields, and more, to match your specific use case.

### 3. Receive the "Split Japanese documents into RAG chunks with overlap" output

Monitor the progress in real time. You will be notified as soon as your dataset is complete and ready for review. Each result includes data such as document index, chunk index, characters, and more.

### 4. Integrate into your workflow

The final output is delivered in JSON, CSV, or Excel format, ready to be plugged into your workflow. You can also [run the "Split Japanese documents into RAG chunks with overlap" task via API](https://apify.com/quietworkslab/japanese-text-normalizer.md).

## FAQ

### Where does the Japanese RAG chunking task split the text?

The Japanese RAG chunking task splits after 。！？ and at line breaks, never inside 「」 or （）. Only a sentence longer than the chunk size is cut, preferably after 、 or a space.

### Which chunk size should I use?

The Japanese RAG chunking task accepts 200 to 100,000 characters per chunk, with an overlap of up to a third of the chunk size. This example uses 200 with an overlap of 40; choose the size your embedding model and retrieval work best with.

### Can the chunks keep the source URL and title?

Yes. When you read from a dataset, the Japanese RAG chunking task copies the metadata fields you choose, such as url and title, into every chunk row.

## About this Actor

This example demonstrates how to use [Japanese Text Normalizer & RAG Chunker](https://apify.com/quietworkslab/japanese-text-normalizer.md) with a specific input configuration. Visit the [Actor detail page](https://apify.com/quietworkslab/japanese-text-normalizer.md) to learn more, explore other use cases, and run it yourself.


## How to integrate an Actor?

This Task's input is already configured above. Use it as-is rather than inventing a new one.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For full API examples (JavaScript, Python, CLI, MCP, OpenAPI), see this Task's Actor page: https://apify.com/quietworkslab/japanese-text-normalizer.md

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).