Arabic Transcription Guidelines for ASR: Dialects, Code-Switching & Tags

Arabic transcription guidelines for ASR, in short: keep a verbatim master transcript and derive the clean version from it. Spell dialect words to a fixed convention, such as CODA, backed by a project lexicon. Write English and French words in Latin script, and write numbers as words. Mark fillers, noise, overlap and personal data with a small closed set of tags, and cut segments of 30 seconds or less, each with one speaker label and timestamps.

Each rule exists because dialectal Arabic has no official spelling, so transcripts drift. In the MGB-3 challenge, four transcribers were left to spell Egyptian Arabic as they saw fit. Even after normalizing three commonly confused letters, the organizers still measured about 13% word-level disagreement between them. That becomes label noise and inflated WER. The CODA* paper opens with 27 spellings found online for a single Egyptian word meaning "he doesn't say it", which CODA writes ما بيقولهاش (ma biʔulhāš). Below are the conventions we recommend, drawn from published Arabic speech corpora.

Verbatim or clean transcription for training data?

For ASR training data, transcribe verbatim: every word as spoken, including repetitions, false starts and fillers, each marked consistently. A clean transcript reads better, but it teaches the model to drop audio it actually heard. The Casablanca corpus went further in the same direction: its annotators were encouraged to write tanween and hamza so that transcripts represent the speech signal faithfully.

You do not have to choose. Make the verbatim transcript the master layer and generate the clean layer from it by rule: drop tagged fillers, collapse repetitions, turn number words into digits. The reverse is impossible: a filler that was never written down cannot be recovered.

How to spell dialect words consistently: CODA and the alternatives

Published Arabic speech corpora have used three strategies for dialect spelling:

For production training data, we recommend the third. CODA (Habash, Diab and Rambow, LREC 2012) set out a conventional orthography for dialectal Arabic, with detailed guidelines for Egyptian. CODA* (Habash et al., LREC 2018) unified them across 28 city dialects from Rabat to Muscat, and explicitly considers dialect text produced as output, such as speech recognition output. The CAMeL guidelines document the rules. The ones transcribers hit most:

• Spell by root, not by sound, when the sound change is regular. In CODA*, ق can stand for /g/, /ʔ/ and other reflexes, so a Gulf speaker's agdar ("I can") is written with ق (أقدر), not with a phonetic letter such as گ. Pronunciation, if needed, goes in a separate layer.
• Attach single-letter future particles. The Egyptian ه+ joins the verb: هبعتلك (habʿatlak, "I'll send you"). Multi-letter particles such as Levantine رح stay separate.
• Write the negation ما as its own word, as in ما بيقولهاش, apart from the listed exceptions.
• Always write word-final taa marbuta ة, however it is pronounced.
• Decide on hamza explicitly. CODA* normalizes word-initial hamzas; ZAEBUC-Spoken writes them when audibly pronounced. For speech we recommend the ZAEBUC choice.

CODA settles the general rules; individual words need a project lexicon, a shared, versioned list of accepted spellings for frequent dialect words, brands and places. Transcribers look words up before inventing a spelling, and a lead approves new entries. Our playbook for annotating Arabic text covers the matching normalization policy for text, and our explainer on MSA vs dialectal Arabic covers why varieties need separate treatment.

Code-switching transcription in Arabic: Latin script or transliteration?

Arabic speech often switches into English or French mid-sentence; in ZAEBUC-Spoken, Arabic-English code-switched utterances make up 19.0% of all utterances that are not annotation-only. There are three ways to write the foreign words:

• Latin script, inline. ZAEBUC-Spoken writes English words in Latin script and joins Arabic clitics to them with a plus sign, as in و+implement+ي. The language boundary stays visible.
• Arabic-script transliteration. QCRI's transcription rules transliterate foreign words, tag them with an ISO 639-1 language code, and keep a shared bank of transliterations for consistency.
• Both. Casablanca asked for each code-switched transcript in both Arabic and Latin script.

Our recommendation: use Latin script in the master layer, joining Arabic clitics with a plus sign, as in ال+app ("the app"). Add an Arabic-script layer only if your product must output all-Arabic text. If a project mixes English and French, tag the French spans, because script alone will not separate them.

Numbers, dates and names

Numbers and dates: write them as words in the verbatim layer, as ZAEBUC-Spoken and Casablanca both do. Casablanca gives the reason: Arabic numbers inflect, so عشرين and عشرون ("twenty") are different spoken forms that digits would erase. Convert to digits in the clean layer with a deterministic script.

Names: spell each person, place and brand the same way throughout, as the QCRI rules require; the lexicon enforces it. Personal names are also personal data: replace them with a tag such as [NAME]. Voice carries its own obligations under laws such as Saudi Arabia's PDPL (see our PDPL data residency checklist).

Disfluencies and fillers in Arabic verbatim transcription

Separate fillers from discourse markers. Fillers are sounds with no lexical meaning, such as امم or اه. Mark them from a closed list with a % prefix, as ZAEBUC-Spoken does. Discourse markers such as يعني (yaʿni, "I mean"), والله (wallāh) and خلاص (khalāṣ, "okay, that's it") are words: transcribe them, because dropping them changes what was said.

Write cut-off words as heard and mark the cut with -- (أحو-- أحوّل). Keep repetitions. Never correct grammar or swap dialect words for MSA.

Non-speech tags for Arabic transcripts

Use few tags, define them all, and make them machine-parseable. This is the set we recommend: the % filler prefix follows ZAEBUC-Spoken, and language tagging follows the QCRI idea of marking foreign spans with an ISO 639-1 code. Never add a tag mid-project without versioning the guideline.

Two illustrative transcripts, tagged

Both examples are illustrative, written for this guide, not client data. First, a Gulf (Saudi) banking call:

Translation: "Hello, how can I help you?" / "Um, I want to trans-, transfer two thousand riyals, but the app won't open, and the code didn't arrive on my number." Note the root spelling of أقدر, the number in words and the Latin-script "app".

Second, an Egyptian telecom call with cross-talk:

Translation: "I want to change my package; the internet has been down since the morning and the (router?) isn't working." / "Certainly, sir, I'll send you a message now." Note the uncertain word and the attached future particle in هبعتلك.

Segmentation, timestamps and speaker labels

Published Arabic corpora keep segments short. QCRI's rules cap them at 15 seconds with at most one second of silence inside. Casablanca required one speaker per segment, at least three words and no more than 30 seconds. ZAEBUC-Spoken sets a preferable minimum of 10 seconds and a maximum of 30 seconds per utterance, unless a segment cannot be divided. Whisper was trained on 30-second audio chunks, so longer segments must be re-split before fine-tuning a Whisper-family model.

• Cut at pauses and clause boundaries, never inside a word, with a little silence at both edges.
• Timestamp every segment (start and end, in seconds, 10 ms precision) against the audio file, not a subtitle track.
• One speaker label per segment: roles on calls (AGENT, CUSTOMER), otherwise anonymised IDs (SPK1, SPK2), with the ID map kept in restricted metadata.
• Handle overlap explicitly: each speaker gets their own segment and timestamps, flagged [overlap]. Never merge two voices into one line.
• Per-segment metadata: variety confirmed by a native listener, channel, code-switch flag.

QA for Arabic transcripts

1. Normalize, then score. Apply your normalization script and lexicon before comparing, as MGB-3 did for letter variants; otherwise you measure spelling, not listening.
2. Seed gold clips into every transcriber's queue and track WER against them by dialect.
3. Double-transcribe a sample independently and send disagreements to a lead for adjudication.
4. Turn recurring disagreements into rules: a lexicon entry or a versioned, dated guideline change.
5. Validate tags mechanically. A script rejects unknown tags, unbalanced brackets, overlapping timestamps and over-length segments before human review.

Be careful automating metadata. In LabelBench, a local language model reached 22.4% exact-match accuracy across five dialect regions from text, on the fixed held-out manifest (chance is 20%). Dialect labels need native listeners. For the wider pipeline, see our guide to Arabic data labeling; for text tasks, adapt our Arabic annotation guidelines template.

Arabic transcription guidelines: a one-page style guide to copy

1. Layers: a verbatim master; a clean layer derived by script.
2. Script: Arabic speech in Arabic script; English and French in Latin script, with Arabic clitics joined by +. No Arabizi.
3. Spelling: CODA* plus the project lexicon. No grammar correction, no conversion to MSA.
4. Diacritics: none, except hamza, tanween and shadda where audible and listed in the lexicon.
5. Numbers and dates: words in verbatim; digits only in the clean layer.
6. Names and acronyms: one spelling per entity; letter-by-letter acronyms joined with underscores (U_A_E), as in ZAEBUC-Spoken; personal data replaced with [NAME], [PHONE] and [ID].
7. Disfluencies: fillers from a closed % list; discourse markers written as words; cut-offs marked with --; repetitions kept.
8. Tags: only those in the tag table.
9. Punctuation: full stop, comma, question mark and ellipsis only (، ؟ in Arabic runs).
10. Segments: 30 seconds or less, one speaker, cut at pauses; start and end timestamps; overlap flagged.
11. Metadata: variety confirmed by a native listener, channel, code-switch flag, guideline version.
12. Change control: every rule change versioned, dated and applied to the gold set.

How Bayanat Labs fits in: our Arabic speech data and transcription service covers transcription, timestamping and diarization of Arabic audio by vetted native speakers and linguists. Contributors pass dialect and domain screening before touching client data, work is calibrated against gold standards, and disagreements go through adjudication and rework loops, with an audit trail on every task.