Text Normalizer avatar

Text Normalizer

Pricing

$0.01 / actor start

Go to Apify Store
Text Normalizer

Text Normalizer

Pricing

$0.01 / actor start

Rating

0.0

(0)

Developer

Eichan Jolly

Eichan Jolly

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

6 days ago

Last modified

Categories

Features

Limited permissions

Share

Actor slug: text-normalizer
Store category: AI
Runtime: Python 3.12 standard library only; no Apify SDK, login, external LLM, or third-party AI service.

What it does

Clean up one text string, reproducibly, before indexing, comparison, or downstream processing. The Actor writes one Dataset item containing the text, its character counts, and an honest change log.

Who it is for

Developers preparing text for search, RAG ingestion, document pipelines, and repeatable text comparisons. This is a small formatting utility, not paraphrasing or AI rewriting.

Method (exact order)

  1. Apply Unicode NFC (canonical composition), without compatibility transliteration.
  2. Replace curly single and double quotation marks (U+2018–U+201F) with ' or ".
  3. Replace en dash U+2013 and em dash U+2014 with -.
  4. Replace each run of ASCII spaces and/or tabs with a single ASCII space.
  5. Trim Unicode whitespace at the beginning and end of each line.
  6. Replace each run of three or more consecutive LF newlines with two newlines.
  7. Trim Unicode whitespace around the entire result.

A changes entry appears only if that operation actually changed the text; counts identify affected quote/dash characters, whitespace sequences, lines, removed newlines, or trimmed outer characters. NFC is recorded as applied without claiming a count of distinct glyph edits. If the input is already normalized, output is byte-for-byte the same string (as UTF-8 JSON), and changes is [].

Input

Required text (string): 1–100,000 Unicode code points; whitespace-only strings are rejected. The prefilled Actor input schema contains the working sample below.

Output fields

normalizedText (string or null on error), charactersBefore (integer or null), charactersAfter (integer or null), changes (list of plain-language strings), status (success or error), and error (null or readable message). Character counts are Python Unicode code-point counts, not UTF-8 byte lengths.

Limits and cautions

  • This is character/whitespace normalization, not grammatical editing, language detection, translation or deduplication.
  • NFC does not perform NFKC compatibility folding (e.g., it does not flatten full-width letters).
  • Straightening quotes/dashes can remove typographic distinctions. Whitespace collapsing can affect code or alignment; avoid using this tool on source code or layout-sensitive text.
  • It does not fetch webpages. No HTTP call is made unless Apify storage REST is needed on-platform.
  • Invalid input returns one Dataset item with status: "error" and a plain-language message; the process exits nonzero after saving it.

Working input example (also the exact schema prefill)

{
"text": "Café — “Hello,” she said.\t\tWe’re ready.\n\n\nNext line. "
}

Actual output example (from script-mode smoke run)

{
"normalizedText": "Café - \"Hello,\" she said. We're ready.\n\nNext line.",
"charactersBefore": 57,
"charactersAfter": 50,
"changes": [
"Unicode normalized to NFC",
"smart quotes straightened: 3",
"em/en dashes replaced with hyphens: 1",
"space/tab sequences collapsed: 4",
"lines with edge whitespace trimmed: 1",
"extra consecutive newlines removed: 1"
],
"status": "success",
"error": null
}

How to run locally

From this Actor folder with Python 3.12+:

# Bash/macOS/Linux: reuse the proven sample input.
ACTOR_INPUT_JSON="$(cat tests/sample_input.json)" python src/main.py

Or create storage/key_value_stores/default/INPUT.json containing the input JSON, then run python src/main.py. Local Dataset items appear in storage/datasets/default/000000001.json (with increasing sequence numbers). You can override the storage root with APIFY_LOCAL_STORAGE_DIR.

Apify platform behavior (no SDK)

The Actor selects input in this order:

  1. ACTOR_INPUT_JSON (when set, even if it contains invalid JSON; invalid input is reported instead of silently falling through).
  2. When APIFY_DEFAULT_KEY_VALUE_STORE_ID exists: GET https://api.apify.com/v2/key-value-stores/{id}/records/INPUT with Authorization: Bearer {APIFY_TOKEN}.
  3. Otherwise: local ./storage/key_value_stores/default/INPUT.json (or the APIFY_LOCAL_STORAGE_DIR equivalent).

On the platform, one result is appended to POST https://api.apify.com/v2/datasets/{APIFY_DEFAULT_DATASET_ID}/items using the runtime token. The Actor does not require customers to provide an API key or log in to a third-party service. Both storage requests have a 20-second timeout; network, HTTP, and TLS certificate problems return readable errors without tracebacks. No custom API base URL validation is imposed.

Reproducible verification

  • python -m unittest discover -s tests -v runs included automated tests, including direct subprocess script mode and local REST mocks.
  • python tests/run_smoke.py re-runs the actual python src/main.py with the schema prefill, saves a local Dataset record, and regenerates tests/smoke_evidence.md, tests/smoke_stdout.txt, and tests/smoke_dataset_item.json.
  • tests/unittest_terminal.txt and tests/smoke_evidence.md contain the captured outputs from the delivery verification. Live Docker builds and Apify-platform deployment were not executed.

Store category and suggested pricing

Category: AI (data preparation utility). Suggested pricing: USD $0.005 per completed run; suggestion only. It is recorded in store_metadata.json. No publishing or billing setup was performed.