# Arabic TTS inspection rubric (v0.1)

Bayanat Labs, CC BY 4.0. Use it to check an Arabic text-to-speech voice, or a set of studio recordings, for pronunciation and speaking style before launch or before a full recording run.

## 1. What you need

- `script.jsonl`: 18 Arabic test lines. Each has the plain `text`, a `reference_reading` with diacritics on the tested words, the `tested_words`, the `error_codes` it can produce and a `listen_for` note in English.
- A written **register and reading policy** for your product: MSA or dialect, which dialect, which calendar and how dates, month names and times are read (TK11), how abbreviations are expanded (TK09), how English terms are said (TK16). Several lines only have a right answer once this policy exists.
- A **style list**: the labels your voice must produce (this kit uses neutral, empathetic, cheerful and urgent).
- One or more listeners who speak the target variety natively. Dialect lines (TK13 to TK16) need a listener from that variety.

## 2. Listening protocol

1. **Give the system the plain `text`, never the `reference_reading`.** Production text arrives without diacritics; the test is whether the voice chooses the right vowels on its own. If you are inspecting human studio takes instead, give the speaker the plain text as well.
2. **Generate or record once per line and style.** TK17 is read twice (neutral, empathetic). Do not cherry-pick: keep the first output, or fix a seed and record it.
3. **Listen with the reference reading in front of you,** at normal speed, then once more for the tested words only.
4. **Write what you heard** in `transcript_as_spoken`, fully vowelled.

A note on the reference readings: a sukun on the last letter marks a pausal form. Final tanwin fath (-an) keeps its standard spelling (صَبَاحًا, غَدًا, مَسَاءً) but is read as a long a at a pause (sabaha, ghada, masa'a).
5. **Mark each tested word** pass or fail. For each failure add an error with a code from section 3, the word, and what was heard.
6. **Score the style** 0 to 2: 0 wrong style, 1 recognizable but flat or overdone, 2 matches the label.
7. **Set the result:** `pass` (no errors, style 2), `fail` (any error in a tested word, or style below 2) or `rerecord` (a human take with a reading slip that a retake will fix).

## 3. Error codes

| Code | What went wrong | Example line |
|---|---|---|
| `SUN_ASSIM` | The lam of the article is heard before a sun letter (al-shams instead of ash-shams) | TK01, TK03 |
| `MOON_LAM` | The lam is dropped or assimilated before a moon letter | TK02 |
| `WASL` | A connecting hamza is pronounced as a glottal stop in connected speech, or a long vowel is not shortened before it | TK03, TK04, TK10 |
| `PAUSAL` | A case ending or tanwin is pronounced at a pause where the pausal form is expected | TK01, TK08, TK17 |
| `TA_MARBUTA` | Ta marbuta read as t at a pause, or silent in a construct phrase | TK04 |
| `GEMINATION` | A doubled consonant is not lengthened (darasa for darrasa) | TK05 |
| `HOMOGRAPH` | The wrong word chosen for an undiacritized form ('ilm for 'alam) | TK06 |
| `NUM_GENDER` | A number from 3 to 10 has the wrong gender for its noun | TK07, TK08 |
| `NUM_VERBALIZE` | A digit, price or time is read wrongly, skipped or read digit by digit | TK07 to TK11 |
| `ABBREV` | An abbreviation is spelled out or expanded wrongly (ر.س, د.) | TK09, TK10 |
| `DATE_POLICY` | The date, calendar, month name or time does not follow your written policy | TK11 |
| `PROSODY_QUESTION` | A question is read with statement intonation | TK12 to TK15 |
| `DIALECT_LEXICON` | A dialect word is replaced by, or pronounced like, its MSA form | TK13 to TK15 |
| `QAF_REALIZATION` | Qaf realized against the variety (q where Najdi speech has g) | TK15, TK16 |
| `CODESWITCH` | English terms mispronounced, read letter by letter in Arabic, or the Arabic around them loses its accent | TK16 |
| `STYLE_MISMATCH` | The delivery does not match the style label | TK08, TK17, TK18 |
| `AUDIO_DEFECT` | Clipping, noise, clicks, truncation or a wrong sample rate | any |

## 4. Report

Report the pass rate for each error code, not only one overall score: a voice that fails every `NUM_GENDER` line needs a text-normalization fix, while one that fails `QAF_REALIZATION` needs different data or a different speaker. Keep the failing outputs with the report so the same lines can be checked again after the fix.

Eighteen lines are a smoke test, not a benchmark. Before launch, extend the script with your own domain terms, names, prices, dates and code-switched phrases, using the same fields.

## 5. Rights check before recording or training

Record every permission separately in each utterance record (`rights` in `schema.json`): ASR training, TTS training, creating a synthetic voice of this speaker, redistribution and commercial use, plus permitted users, exclusivity, expiry and a pointer to the signed consent. A permission that is not written down is `not_stated`, never `granted`. Consent to contribute recordings for speech recognition does not by itself permit a reusable synthetic voice of that speaker.