Bayanat Labs Arabic Text Normalizer (arabic-normalize)
arabic-normalize applies a named normalisation policy, such as bayanat-norm-v1, to Arabic text and reports exactly what changed: the normalised text, a change log with the position of every change, the lossy changes, the diacritization ratio and the SHA-256 of the policy. It comes as a Python library, a JavaScript package in the same repository and a free in-browser tool, with conformance fixtures that both implementations pass.
| Licence | Apache-2.0 (code); CC BY 4.0 (conformance fixtures) |
|---|---|
| Version | v0.1.0 · Pilot, not released yet: no release tag, and not on PyPI or npm. Conformance fixtures v0.1. |
| Published | |
| Updated | |
| Who it is for | Teams that normalise Arabic text for search, training data or speech-recognition scoring and need to state exactly which normalisation they used. |
| Level | Intermediate |
Key numbers
Counted by script from the released files. Each key number is read from MANIFEST.json in the Hugging Face conformance dataset and pinned to its SHA-256.
Downloads and identifiers
| Where | What |
|---|---|
| bayanatlabs.com | Open the normalizer |
| GitHub | Code on GitHub |
| Hugging Face | Conformance fixtures on Hugging Face |
| Hugging Face | In-browser tool on Hugging Face Spaces |
Get started
Install from GitHub, pinned to a commit (the packages are not on PyPI or npm yet)
# Python 3.9 or later
pip install "git+https://github.com/Bayanat-Labs/arabic-normalize@3ad22326a66596e8dc413321017251eed6e60439"
# JavaScript (Node 20 or later): the js/ folder of a clone at the same commit
git clone https://github.com/Bayanat-Labs/arabic-normalize.git
git -C arabic-normalize checkout 3ad22326a66596e8dc413321017251eed6e60439
npm install ./arabic-normalize/jsWhat is released
| Part | Licence | In this release |
|---|---|---|
| Python library, CLI and JavaScript package | Apache-2.0 | Yes |
| Conformance fixtures (dev and acceptance splits) | CC BY 4.0 | Yes |
| In-browser normalizer on Hugging Face Spaces | Apache-2.0 | Yes |
Attribution: arabic-normalize by Bayanat Labs. https://github.com/Bayanat-Labs/arabic-normalize
Intended uses
- Normalise Arabic text with a named, versioned policy and record its SHA-256 in a paper, a score or a dataset card.
- See what a normalisation changed, with the position of every change and the lossy flags.
- Test another implementation against the conformance fixtures.
Out of scope
- Spelling correction or diacritization: the policies remove or map characters; they do not restore them.
- Changing a released policy: any change in behaviour ships under a new policy name.
Rights at a glance
Training, display, redistribution and synthetic-voice use are separate permissions.
| Use | Status | Note |
|---|---|---|
| Training | Allowed | Code under Apache-2.0; fixtures under CC BY 4.0 with attribution. |
| Display | Allowed | Fixtures under CC BY 4.0, with attribution. |
| Redistribution | Allowed | Code under Apache-2.0; fixtures under CC BY 4.0 with attribution; Unicode data under the Unicode License V3. |
| Synthetic voice | Not applicable | Text only. |
Limitations
- Install from GitHub for now; the PyPI and npm packages are not published yet.
- The conformance fixtures cover the three released policies only.
Sources and licences
| Source | Licence | Redistributed here |
|---|---|---|
| Unicode Character Database 18.0.0 | Unicode License V3 | Yes |
Written by Bayanat Labs. The library bundles data derived from the Unicode Character Database, covered by the Unicode License V3 (see NOTICE in the repository). Source details
How to cite
Cite as:
Bayanat Labs (2026). arabic-normalize: Arabic text normalisation with named, versioned policies, version 0.1.0. Bayanat Labs. https://github.com/Bayanat-Labs/arabic-normalize
No DOI. Cite the URL in the citation above and the date you read it.
@misc{bayanatlabs2026arabicnormalize,
author = {{Bayanat Labs}},
title = {arabic-normalize: Arabic text normalisation with named, versioned policies, version 0.1.0},
year = {2026},
month = oct,
howpublished = {\url{https://github.com/Bayanat-Labs/arabic-normalize}}
}
Changelog
| Date | Version | Change |
|---|---|---|
| v0.1.0 | Pilot code (v0.1.0, not released yet) on GitHub, the in-browser tool on Hugging Face Spaces and the conformance fixtures on Hugging Face. |
Corrections and removal requests
If something in this release should not be there, or is wrong, email hello@bayanatlabs.com. We handle removal requests within 30 days, and platform requests within 24 hours. How removal requests work
Related product: Bayanat Bench