How to Annotate Arabic Text: A Practical Playbook for NLP Teams

To annotate Arabic text well, settle five decisions in writing before anyone labels anything: how words are split into clitics, which character variants are normalized, how register and dialect are tagged, where spans start and end, and what happens to Arabizi and English or French inside Arabic. Then annotate the raw, unnormalized text with character offsets, pilot on a small batch seeded with gold items, and feed every adjudicated disagreement back into the guideline. Most Arabic label noise comes from leaving these choices to each annotator, not from annotators misreading Arabic.

The academic groundwork exists: orthography conventions, treebank tokenization schemes, toolkits such as CAMeL Tools. This playbook turns it into defaults for Arabic NLP annotation that a product team can adopt this week. For task types, taxonomy design, cost and staffing, start with our complete guide to Arabic data labeling.

The five decisions that change every Arabic label

In Arabic text annotation, each decision produces a different dataset from the same raw text. Left unwritten, each annotator makes it silently, and the disagreement surfaces later as "noise".

Tokenization and clitics: do you annotate words, stems or clitics?

Arabic attaches conjunctions (و، ف), prepositions (ب، ل، ك), the definite article (ال), the future marker (س) and pronoun suffixes directly to words. وبمدرستهم (wa-bi-madrasati-him, "and in their school") is one whitespace token and four morphemes, و + ب + مدرسة + هم, and the stem's final ة surfaces as ت once the suffix follows. The same density drives the Arabic tokenization tax in English-first LLMs.

There is no single standard segmentation. CAMeL Tools' morphological tokenizer documents several schemes. In its own example, فتنفست الصعداء (fa-tanaffasat aṣ-ṣuʿadāʾ, "so she sighed with relief"), the D3 scheme splits off the article while the ATB (Penn Arabic Treebank) scheme keeps الصعداء whole. Labels made on one scheme's tokens inherit that scheme. Character offsets on the raw string keep the choice open: you can project spans onto D3, ATB or any subword tokenizer later.

Your tool often makes this choice for you:

• Token snapping captures clitics. Prodigy's NER documentation explains that selections snap to token boundaries, so selecting part of a word locks in the whole word. INCEpTION's user guide says token-level granularity expands a smaller selection to the covering token. With whitespace tokens, tagging the city in وبيروت (wa-Bayrūt, "and Beirut") stores the conjunction too. Both tools offer character-level selection instead (Prodigy's --highlight-chars flag; INCEpTION's character granularity).
• The canonical form may not be in the string. In بالرياض (bi-r-Riyāḍ, "in Riyadh") the article is intact. In للرياض (li-r-Riyāḍ, "to Riyadh") its alef drops, and الرياض occurs nowhere in the text. Our rule: the span starts after the preposition (لرياض), and annotators record الرياض as the canonical form in an attribute.
• Pronoun suffixes need a rule. For اشتراكي (ishtirākī, "my subscription"), decide whether a service slot covers اشتراك or the whole word. Excluding the suffix keeps the slot comparable across dialects, as the worked examples show.

Arabic annotation normalization policy: normalize or preserve?

The rule that saves the most rework: the raw text is immutable, offsets always refer to it, and normalization is a derived view for search, deduplication, lexicon matching and model input. Normalizing first changes character counts. Stripping diacritics or tatweel shortens the string, and Unicode NFKC expands the single lam-alef ligature ﻻ (U+FEFB) into two letters, so earlier offsets point at the wrong characters.

CAMeL Tools (Obeid et al., LREC 2020), an open-source Arabic NLP toolkit, has a normalization module with one function per variant (alef, alef maqsura, taa marbuta) plus NFC/NFKC normalization, and a separate dediacritization module, so you can decide each row below on its own.

The two "no" rows are the exception: fix presentation forms and Persian lookalikes in one versioned clean-up pass before annotation, so offsets are computed on clean text from the start. Arabic-Indic digits (٢٠٢٦ vs 2026) stay as written; your date and amount rules must accept both.

How should you tag register and dialect on every item?

Tag each item, not the dataset. A "Gulf customer service" corpus still contains MSA boilerplate, Egyptian agents and Levantine customers. Four item-level fields cover most needs:

• Register: MSA, dialectal, or mixed.
• Variety, at the coarsest level your product needs. The MADAR corpus paper (Bouamor et al., LREC 2018) notes that dialects are often classified by region (Egyptian, North African, Levantine, Gulf, Yemeni) or sub-region (Tunisian, Lebanese, Kuwaiti and so on); MADAR itself covers 25 city dialects. Most products need region plus country.
• Script: Arabic, Arabizi, or mixed.
• Confidence, with "undetermined" as a legal value. شكرا جزيلا (shukran jazīlan, "thank you very much") carries no dialect marker, and forcing a guess teaches the model noise.

Dialect is where cheap automation fails most visibly. On LabelBench v0.1, Qwen 2.5 3B reached 22.4% on five-region dialect identification, where a balanced guess scores 20%, and labeled 75% of Gulf examples as Egyptian. Route dialect tags to native speakers of that variety, and read our analysis of MSA versus dialect coverage before choosing which varieties to include.

Worked examples: one intent in MSA, Gulf, Egyptian, Levantine and Maghrebi

One intent, cancel_subscription, in five varieties. The sentences are illustrative, written for this guide; real spellings vary (ابغى without hamza, عاوز for عايز). Every row means "I want to cancel my subscription."

• The label never changes; the evidence does. Five different "want" words carry one intent. The Gulf and Moroccan forms share a root (b-gh-y) yet look nothing alike on the page; keyword pre-labels and MSA-only annotators miss them.
• Possession moves out of the word. MSA, Gulf and Levantine attach "my" as a suffix; Egyptian بتاعي and Moroccan ديالي are separate words. Excluding possessive suffixes makes the slot the same noun in every row, article or not.
• Moroccan نلغي means "I cancel", not "we cancel". In MSA the prefix ن marks "we"; in Moroccan Arabic it marks the first-person singular. Anyone tagging grammatical person gets this wrong unless the guideline says so.

When annotators write text themselves (normalized forms, corrections, transcriptions), give them a spelling standard. CODA (Habash, Diab and Rambow, LREC 2012) defined conventional orthography for dialectal Arabic, starting with Egyptian. The unified CODA* guidelines extend it across dialects, for example attaching single-letter future particles such as ح+ and ه+ to the verb and always writing word-final taa marbuta. Apply CODA to what annotators write, never to the source they label.

Arabizi, English and French inside Arabic text

Arabizi is Arabic in Latin characters, with digits standing in for letters that have no Latin equivalent, such as 2 for hamza and 3 for ʿayn. Darwish (2014) notes it is used for both MSA and dialects and is often mixed with English or French. Customer data mixes all of these:

• Two scripts in one line: ممكن تبعتلي الـ invoice؟ (mumkin tibʿatli el-invoice?, Egyptian, "can you send me the invoice?"), where a tatweel lets the article stand before a Latin word.
• French inside Darija: فين la commande ديالي؟ (fīn la commande dyāli?, Moroccan, "where's my order?").
• Arabizi: 3ayez alghi el eshterak (Egyptian) and baddi 2alghi l ishtirak (Levantine), both "I want to cancel the subscription".
• Hybrid words: in the illustrative bghit n-annuler l'abonnement dyali, a Moroccan first-person prefix sits on a French verb, and no single language tag fits.

You do not need to invent a token-level language scheme. The first shared task on code-switched language identification (Solorio et al., 2014) included an MSA–dialectal Arabic pair and used six labels: lang1, lang2, mixed, ambiguous, named entity and other. Its "mixed" label covers exactly the hybrid case, such as a dialectal future prefix on an MSA stem. Extend it to Arabic, English and French and add a script attribute.

Two rules keep Arabizi usable: label intent, sentiment or entities on the original text as written, and if you need Arabic-script output, run transliteration as a separate reviewed task with its own gold items. Folded into labeling, every annotator converts differently.

How do you set up an annotation tool for right-to-left Arabic text?

Text is stored in logical order (the order it was typed); the Unicode Bidirectional Algorithm reorders it for display. Offsets count logical order. Mix in Latin words, digits and punctuation, as most real data does, and a selection can look as if it ends where it does not.

The W3C's guidance on HTML direction gives two rules that matter here: set base direction with the dir attribute in markup, not CSS, and remember that dir="auto" takes direction from the first strongly directional character. iPhone 15 مش شغال (Egyptian, "iPhone 15 isn't working") starts with a Latin word, so an auto-direction interface lays the line out left-to-right and left-aligned, with trailing punctuation at the wrong end. What the tools themselves document:

Whatever the tool, run a pre-flight on 20 real items before anyone is paid to label:

1. Include mixed Arabic and Latin, digits, emoji, diacritics, tatweel, and a line that starts with a Latin word.
2. Check that lines are right-aligned and that ؟ lands at the correct end.
3. Annotate one span per mixed line, export, and compare the offsets with a Python slice of the raw string. They must match exactly.
4. Watch emoji: Arabic letters and diacritics count as one unit in both JavaScript and Python, but most emoji count as two UTF-16 units in JavaScript and one code point in Python, shifting every later offset.
5. Check whether import trims whitespace or normalizes Unicode, and whether the font places diacritics correctly.

The QA loop: pilot, gold items, adjudication, revisions

• Pilot with overlap. Several annotators label the same small batch. Measure chance-corrected agreement, then sort each disagreement by cause: segmentation, normalization, dialect, span boundary or genuine ambiguity. Most trace back to the five decisions.
• Build gold items that test the policy. Seed items that hit each rule: للرياض, على versus علي, the Moroccan ن prefix, an Arabizi line, a mixed-script line, and an item whose right answer is "undetermined". Hide gold in production batches, not only in training.
• Adjudicate by variety. Gulf disagreements go to a senior Gulf-native reviewer, not whoever is free. Each resolution becomes a guideline example.
• Version everything. Stamp each label with its guideline version, so a rule change means reworking only the affected items.
• Report quality by slice. Agreement and gold accuracy per variety and per decision; an overall number can look healthy while one dialect fails.

This is the loop we run at Bayanat Labs: Vet, Train, Produce, QA, Deliver. Contributors pass dialect and domain screening before touching client data, work is calibrated against gold standards with adjudication and rework loops, and every task has an audit trail. Our Arabic data annotation services cover NER and spans, intent, sentiment, transcription and segmentation; where a specific variety is the bottleneck, our Arabic dialect data work draws on native speakers across 25+ varieties. Comparing providers? Our vendor checklist and scorecard turns this loop into questions.

How to annotate Arabic text: your first 500 items, step by step

Five hundred items is a practical first milestone, not a statistical rule: enough to expose guideline gaps in every variety before rework gets expensive. The split below is illustrative.

1. Freeze the raw text. Run the one-time clean-up, version the script, hash each file.
2. Write the five decisions on one page, with three Arabic examples per rule, including a counter-example.
3. Stratify the sample, for example 100 items per variety, drawn from real traffic rather than translated English.
4. Configure the tool for character-level spans and right-to-left display; run the 20-item pre-flight.
5. Calibrate on 50 items with three annotators each, adjudicate every disagreement, publish guideline v1.1.
6. Promote adjudicated items to gold, covering every rule and variety.
7. Label the remaining 450 with double annotation on a share of items and hidden gold in every batch.
8. Review by variety and decision, reworking only items touched by guideline changes.
9. Train a quick baseline and read its errors. Confusions clustered around one rule usually mean the rule is unclear.
10. Freeze the guideline and budget the scale-up with our guide to Arabic annotation cost drivers.