Unicode Confusable Codepoint Lookup for AI Agents

Look up hexadecimal codepoints in official Unicode UTS #39 data and return direct confusable mappings with the published security-data version.

Use this actor as a data step in an identifier-validation pipeline. It downloads the official confusables.txt, preserves ordered multi-codepoint targets, and returns an explicit unmapped result when the table has no source entry. An unmapped codepoint does not prove that an identifier is safe.

What it returns

Field	Meaning
inputIndex	Zero-based input slot; order and duplicates are retained.
sourceCodepoint	Uppercase hexadecimal scalar, padded to at least four digits; null for invalid input.
targetCodepoints	Direct target sequence from the data file, in order, retaining repeated codepoints. Empty for unmapped/error rows.
skeleton	Characters made by joining that direct target sequence; null for unmapped/error rows.
mappingType	Original third data-file field. Current Unicode data uses MA; this field is not a risk score.
unicodeSecurityDataVersion	Version read from the verified official file header; null for error rows.
sourceUrl	Official version-specific source URL for successful lookups; null for error rows.
status	mapped, unmapped, or error.
error	Redacted code/message for invalid_codepoint or source_unavailable; otherwise null.

The default dataset is the output. The default key-value store also receives SUMMARY with result/status counts, charge attempts, version and charge-limit state. Return strings may contain combining marks or invisible characters; use targetCodepoints when inspecting exact values.

Input example

The Console prefill is:

{"codepoints":["0430","03BF","FF21"],"maxItems":3}

Input	Behavior
codepoints	Required array. Each entry is one scalar in 1–6 hexadecimal digits, optionally prefixed with U+ or 0x. Case and surrounding whitespace are normalized. Maximum array length: 10000 entries. An empty array produces zero rows.
maxItems	Optional integer, 1–10000; default 1000. Only the first this many input slots are processed, including invalid entries and duplicates. Console prefill: 3.
unicodeVersion	Optional latest (default) or an exact published version such as 18.0.0. An unavailable version produces source errors.

Account names, email addresses, domains, arbitrary personal text, URLs, and Unicode characters entered as text are rejected as codepoint values. Unknown top-level input fields are rejected. Invalid items produce errors without echoing the rejected text; malformed input structure fails with a clear message. The original input can still remain in Apify's run input storage, so submit only codepoints.

Output example

The following row is copied from the real local prefill run; sample_output.json contains all three rows:

{
  "sourceCodepoint": "0430",
  "targetCodepoints": [
    "0061"
  ],
  "skeleton": "a",
  "mappingType": "MA",
  "unicodeSecurityDataVersion": "18.0.0",
  "status": "mapped",
  "inputIndex": 0,
  "sourceUrl": "https://www.unicode.org/Public/18.0.0/security/confusables.txt",
  "error": null
}

For 006D, the direct target is ["0072", "006E"] and skeleton is rn. For 0061, there is no source mapping in the tested version: status is unmapped, targetCodepoints is [], and skeleton/mappingType are null.

Source and version cache

Data comes from Unicode's official security files; the UTS #39 data-file format defines source, target and mapping-type fields. It is not Python's bundled Unicode database.

The named Apify key-value store unicode-confusable-lookup-cache persists downloaded data using keys such as confusables-18-0-0. Cached files are checked against their header, requested version and stored SHA-256 digest. Invalid cache entries are downloaded again. Pin unicodeVersion to reuse its cache across runs. latest downloads the current official file every run, reads its version and stores it under that published version. There is no stale-cache fallback when latest cannot be fetched.

Unicode 17 and later use /Public/<version>/security/confusables.txt; earlier releases use /Public/security/<version>/confusables.txt. Version changes can change mappings. Keep the returned version beside stored results and re-evaluate downstream data when changing versions.

Pricing

Proposed event: confusable_lookup, $0.0001 per mapped or unmapped codepoint row. Invalid-input and source-error rows are uncharged. Duplicate input entries count separately. At that proposal, 1000 successful rows cost $0.10 in event fees. Real prices become effective only after the owner configures Apify monetization; this build does not set account pricing.

See PRICING.md for the captured Store comparison, tied usage counts, competitor prices and the difference between billing per string and per codepoint. Local runs attempt the same event calls but do not bill money. Reaching a configured charge limit can stop a run before maxItems.

Limits and error handling

• No entity lookup, account discovery, email/domain query, network security scan, login, proxy input, arbitrary source URL, or LLM verdict.
• This is a direct codepoint mapping lookup. It does not implement the full UTS #39 identifier skeleton algorithm, normalization, bidirectional reordering, script restrictions, or identifier comparison. skeleton names the direct mapped target characters only.
• Scalars must be in 0000–10FFFF, excluding surrogates D800–DFFF. Valid but unassigned scalars can return unmapped.
• One download runs at a time; lookup processing is sequential. Downloads use a 30-second timeout per attempt, at most three attempts with 1- and 2-second backoff for transient failures, and a 5 MiB decoded response cap.
• Missing, malformed or wrong-version official data produces source_unavailable errors for valid requested entries, never unmapped success. Treat error rows and partial runs as incomplete work.
• Real cloud event charging, container build and publication have not been exercised in this local delivery.

Use with AI agents / MCP

After the owner publishes the actor, an agent can run it through the Apify Actor API or the Apify MCP server. Send the small structured input above, read the default dataset, and branch on status before consuming the target sequence. Store unicodeSecurityDataVersion with each downstream record. Use hexadecimal codepoints prepared by the calling application; do not send personal text.

This actor is not itself an MCP server. A calling agent must apply its own identifier policy and additional Unicode checks. Neither mapped nor unmapped is an allow/block verdict.

FAQ

Why can one source return several targets? The official table contains sequences, for example 006D maps to 0072 006E. Flattening this to one target would lose data.

Does unmapped mean safe? No. It says only that this version of the table has no direct source entry for that scalar. It says nothing about an entire identifier.

Why pin a version? Results and caches then refer to the same published file across runs. latest deliberately refreshes the official source.

Can I submit a username, domain or email? No. Use individual hexadecimal codepoint strings only; the actor does not look up entities or accept arbitrary text.

Local development

Dependencies are apify and httpx; tests use the existing pytest. The shared environment is used without installing packages into it.

$env:APIFY_LOCAL_STORAGE_DIR = "$PWD/storage"
$env:APIFY_PURGE_ON_START = "false"
$env:PYTHONUTF8 = "1"
New-Item -ItemType Directory -Force storage/key_value_stores/default | Out-Null
'{"codepoints":["0430","03BF","FF21"],"maxItems":3}' | Set-Content -Encoding utf8 storage/key_value_stores/default/INPUT.json
C:/GritWork/earn/.venv/Scripts/python.exe -m src
C:/GritWork/earn/.venv/Scripts/python.exe -m pytest -q -p no:cacheprovider
apify validate-schema

The test suite includes @pytest.mark.live for the official-source smoke test; use -m "not live" for offline unit tests. With purge disabled, dataset rows accumulate between local runs; use a fresh local storage directory when you need an isolated dataset.

Official Unicode data is distributed under Unicode License V3. The notice is preserved in UNICODE-LICENSE.txt and fixture headers.