# Arabic Text Normalizer (`abdullah-arabic-tools/arabic-text-normalizer`) Actor

Deterministic Arabic text normalization for search or NFC-preserving cleanup. Offline core, configurable digits and whitespace, bounded inputs, no LLM. Non-personal text only.

- **URL**: https://apify.com/abdullah-arabic-tools/arabic-text-normalizer.md
- **Developed by:** [Abdullah Alsmaeel](https://apify.com/abdullah-arabic-tools) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 text normalizeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Arabic Text Normalizer

Deterministic Arabic text normalization for search indexes, non-personal product catalogs, and text-processing pipelines. No LLM, spelling correction, scraping, or external API in the normalization core. Original source; MIT licensed.

Version 0.1.0. Use it to make Arabic search keys consistent without rewriting the display text in your original catalog. Preserve your originals separately: search normalization is intentionally lossy and is not an HTML sanitizer, spelling corrector, or anonymizer.

### Quick start on Apify

1. Open the Actor's Input form. Submit synthetic or public non-personal text only.
2. Set `texts` to `["أَهْلًا بِالْعَالَمِ ١٢٣", "مدرسة ؤ ئ"]`, `profile` to `search`, and `digitMode` to `ascii`.
3. Check the live Pricing tab and Maximum cost per run before starting. The current event price is $0.001 per result; a $0.002 result budget allows at most two results. A smaller budget can return only part of a batch. No extra usage charge or paid startup event is configured.
4. Read the default dataset. The example's normalized `text` values are `اهلا بالعالم 123` and `مدرسة ؤ ئ`. Each row also includes its index, settings, input/output code-point counts, warnings, and tool/Unicode version. Download JSON or CSV using Apify's dataset export.
5. Check the `OUTPUT` summary in the default key-value store. `complete` means all accepted inputs were published; `budget_limited` means only the indicated `publishedCount` were published. Check the dataset before retrying an interrupted or ambiguous run.

#### بداية سريعة بالعربية

الأداة توحّد النص العربي لاستخدامه كمفتاح بحث، ولا تصحح الإملاء ولا تخفي البيانات الشخصية. احتفظ بالنص الأصلي منفصلًا. أدخل نصوصًا اصطناعية أو عامة غير شخصية في `texts`، واختر `search` للبحث أو `conservative` للحفاظ على الحروف والتشكيل مع NFC. اختر `ascii` لتحويل الأرقام العربية والفارسية إلى 0–9 عند الحاجة. السعر الحالي 0.001 دولار لكل نتيجة، حتى إن لم يتغير النص؛ حد التكلفة قد يوقف الدفعة جزئيًا. النتائج في dataset والملخص في `OUTPUT`؛ لا تكرر تشغيلًا غامضًا قبل فحصه. لا تستخدم التطبيع الذي يحذف التشكيل أو يوحّد الحروف على نص ديني أو قانوني معتمد.

### Run locally

Python 3.11+; no third-party package is required for the local core or unit tests. The Unicode database depends on the Python version; output includes `unicodeVersion`. Acceptance here used Python 3.14.7, not every supported Python version.

```
python arabic_normalizer.py < examples/input.json > result.json
python -m unittest discover -s tests -v
```

Input is UTF-8 JSON on stdin; output is JSON on stdout. Success exits 0. Invalid input exits 2 with a fixed `error.code`, `error.path`, and `error.message`, without echoing supplied values. No file paths, URLs, or arbitrary code are accepted as input options. Output is plain text data; consumers must escape it when rendering HTML.

### Input and exact behavior

- `texts`: required array of 1–100 strings. Each string has 1–8192 Unicode code points, and the batch has at most 65536 code points. The CLI accepts at most 1048576 UTF-8 JSON bytes. Whole-batch validation occurs before any result processing or billing. Empty strings and unpaired surrogates are rejected. A whitespace-only string is allowed and can normalize to an empty result.
- `profile`: `conservative` by default. Applies Unicode NFC only; preserves Arabic marks, Tatweel, presentation-form letters, Alef/Yeh distinctions, and digits unless another option explicitly changes them.
- `profile: search`: NFC, then compatibility expansion of Arabic presentation-form characters in U+FB50–U+FDFF and U+FE70–U+FEFC, NFC again, removal of the exact marks listed below and U+0640 Tatweel, then `أ إ آ ٱ` → `ا` and `ى` → `ي`. This is lossy and can merge different words. It does not fold `ة` to `ه`, change `ؤ`/`ئ`/`ء`, translate text, stem, correct spelling, or normalize Persian Kaf/Yeh. Compatibility expansion is restricted to the Arabic blocks; full-width Latin and other scripts are not NFKC-folded.
- `digitMode`: `preserve` by default. `ascii` maps U+0660–U+0669 and U+06F0–U+06F9 to ASCII 0–9. Decimal and percent separators are unchanged; this is not number parsing.
- `whitespace`: `preserve` by default. `collapse` uses Python Unicode whitespace rules to trim both ends and replace every whitespace run with one ASCII space. This removes line breaks and can alter formatting.
- `removeBidiControls`: `false` by default. `true` removes only U+061C, U+200E, U+200F, U+202A–U+202E, U+2066–U+2069. These 12 characters affect visual ordering. ZWJ, ZWNJ, and BOM are not removed. This option is not a security sanitizer.

Search profile removes these explicitly enumerated mark ranges: U+0610–U+061A, U+064B–U+065F, U+0670, U+06D6–U+06DC, U+06DF–U+06E4, U+06E7–U+06E8, U+06EA–U+06ED. It does not claim to remove every Arabic mark in every Unicode extension or preserve religious recitation annotations. Do not use lossy normalization on authoritative legal or religious text.

Example (synthetic): `أَهْلًا بِالْعَالَمِ ١٢٣` with search + ascii + collapse produces `اهلا بالعالم 123`. `مدرسة ؤ ئ` remains unchanged. `examples/input.json` and `examples/output.json` provide a complete runnable example.

### Output

Local output is described by `schemas/output.schema.json` and contains settings, tool/Unicode version, processed count, total input code points, and `results`. Each result has a zero-based `index`, normalized `text`, `changed`, `lossy`, input/output code-point counts, and warning codes. No separate original-text field is retained.

Warnings: `bidi_controls_present`, `bidi_controls_removed`, `control_characters_preserved`, `not_html_sanitized`, `lossy_transformation`. Warnings are not a PII detector or a complete safety assessment. Canonical NFC equivalence alone is reported as changed but not lossy.

Input contract: `schemas/input.schema.json`. Apify UI input contract: `.actor/input_schema.json`. Runtime enforces aggregate limits, strict boolean types and surrogate rejection in addition to the JSON schema. JSON duplicate keys, non-finite constants, invalid UTF-8 and malformed JSON are rejected by the local CLI. The Apify platform parses JSON before the adapter, so duplicate-key detection at that upstream boundary is not claimed.

### Privacy and limitations

Do not submit personal data, customer records, credentials, bank identifiers, or confidential text. This tool does not detect or automatically anonymize personal information. Synthetic or public non-personal text only.

The local CLI does not make network requests, write files, or log inputs. Shell redirection saves exactly the output path you choose. Normalized text may retain all sensitive content from the input; losing a diacritic is not anonymization.

When run on Apify, Apify stores the run input and dataset under its storage/retention settings. The adapter also stores a summary as `OUTPUT`. It does not delete or promise immediate erasure of platform storage. The developer's source does not send text to external services; this is not a promise that cloud infrastructure has no access. Consumers must review Apify privacy/retention settings and permissions.

This is a structural text-processing tool, not tax or legal advice, a ZATCA certification, or a spelling/grammar guarantee. AI-assisted development was used; the runtime itself uses deterministic code, not AI.

### Apify deployment candidate

`apify_main.py` uses the official Apify Python SDK. `requirements-cloud.txt` identifies the SDK and `requirements-lock.txt` pins the resolved 30-package runtime closure. The SDK was exercised in a network-disabled namespace without credentials: full, zero-budget, partial-budget, invalid-input, missing-event, extra-paid-event and non-PPE cases passed real local-storage assertions. These packages are not required for the CLI. Local SDK charging logs are simulations, not live cloud billing or revenue evidence.

The existing cloud build 0.0.1 was additionally checked with four synthetic cases: small batch (2 results), aggregate-invalid input (rejected with no dataset results), maximum aggregate input (100 texts / 65536 code points, stopped at 2 results by a $0.002 event budget), and a 2-text batch limited to 1 result by a $0.001 event budget. This does not establish complete 100-row cloud output, external customer use, payout eligibility, or an exact all-in platform cost. Cloud tests used 256 MB and a 60-second timeout; actual output includes Unicode version 16.0.0.

Use limited permissions, no proxies, no schedules, no Standby. Read only default input and write only default dataset/summary. Each accepted `text-normalized` event corresponds to one output item, including an unchanged result or a whitespace-only input that becomes empty. Invalid batches produce no text result events. An interrupted run may have partial charged results; its dataset is authoritative, not an absent final summary. Do not automatically retry an ambiguous run: read its state and dataset first to avoid duplicate processing/charges.

Configured price: $0.001 per text result through the single `text-normalized` event, no paid startup or default-dataset synthetic event, and no extra usage charge. Check the live Pricing tab before running; it is the current price reference. `apify_main.py` requires PPE configuration and stops at its event budget; no manual repeated charge/retry logic is present. Invalid input is rejected before text-result charging. An unchanged result or a whitespace-only input that becomes empty still counts as one result.

Use the Actor's API options in Apify Console for the account-authorized request examples. MCP discovery and external-agent integration have not been independently verified; they are not guaranteed by this README. For problems with this Actor, use its Apify Issues tab. Do not include private text, credentials, or personal data in an issue.

### License and technical references

Our code/docs: MIT (`LICENSE`). Python standard library follows the Python/Unicode distribution licenses. Optional Apify SDK: Apache-2.0 per official PyPI metadata for 4.1.0. Resolved package license metadata and the OSV query were checked; this is not a full third-party source audit. See `DEPENDENCIES.md`.

- Python Unicode normalization: https://docs.python.org/3/library/unicodedata.html
- Apify PPE: https://docs.apify.com/actors/publishing/monetize/pay-per-event
- SDK charging and limits: https://docs.apify.com/sdk/python/docs/concepts/pay-per-event
- Limited permissions: https://docs.apify.com/actors/development/permissions

Arabic documentation: `README.ar.md`.

# Actor input Schema

## `texts` (type: `array`):

1–100 non-personal text items. 1–8192 Unicode code points each; 65536 in total (runtime enforced).

## `profile` (type: `string`):

conservative: NFC only. search: expands Arabic presentation forms, removes listed Arabic marks/tatweel, folds Alef variants and Alef Maksura. Search is lossy.

## `digitMode` (type: `string`):

ascii maps Arabic-Indic and Eastern Arabic-Indic digits to 0–9; decimal separators are preserved.

## `whitespace` (type: `string`):

collapse strips edges and replaces runs of Unicode whitespace with one ASCII space; removes line breaks.

## `removeBidiControls` (type: `boolean`):

Opt-in removal of the 12 listed Unicode bidi controls. May change visual order; does not sanitize HTML or guarantee safe display.

## Actor input object example

```json
{
  "texts": [
    "أَهْلًا بِالْعَالَمِ ١٢٣",
    "متجر تجريبي"
  ],
  "profile": "conservative",
  "digitMode": "preserve",
  "whitespace": "preserve",
  "removeBidiControls": false
}
```

# Actor output Schema

## `results` (type: `string`):

Default dataset items with normalized text, index, settings, code-point counts, warnings, and tool/Unicode versions. A budget-limited or interrupted run can contain a partial batch.

## `summary` (type: `string`):

Default key-value store OUTPUT record: accepted input count, published count and complete/budget_limited status, or a validation error. An interrupted run may not have a final summary; check its dataset before retrying.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "texts": [
        "أَهْلًا بِالْعَالَمِ ١٢٣",
        "متجر تجريبي"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("abdullah-arabic-tools/arabic-text-normalizer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "texts": [
        "أَهْلًا بِالْعَالَمِ ١٢٣",
        "متجر تجريبي",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("abdullah-arabic-tools/arabic-text-normalizer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "texts": [
    "أَهْلًا بِالْعَالَمِ ١٢٣",
    "متجر تجريبي"
  ]
}' |
apify call abdullah-arabic-tools/arabic-text-normalizer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,abdullah-arabic-tools/arabic-text-normalizer"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/pYWxwVkxw15ikHXwM/builds/lXLflsJdXLuvlw8Ti/openapi.json