The tokenization tax: the hidden cost of Arabic in English-first LLMs

Procurement teams compare models on price per million tokens. Engineering teams budget context windows in tokens. Latency dashboards count tokens per second. All three assume the token is a stable unit of content. For Arabic, it is not — and the difference is large enough to change vendor decisions, architecture choices and unit economics.

The metric that exposes this is fertility: the average number of tokens a tokenizer produces per word. English fertility on mainstream tokenizers sits near 1.3 — most words survive as one piece. Arabic fertility on the same tokenizers has historically ranged from roughly 2 to 4 or more, meaning the identical sentence, translated, can consume two to three times the tokens. Nothing in your invoice will label this; it simply arrives as a higher bill for the same work.

Where the tax comes from

Three properties of Arabic collide with the way subword tokenizers are built:

• Morphology packs a sentence into a word. Arabic attaches conjunctions, prepositions, the definite article and object pronouns directly to the stem. A single written word like وسيكتبونها carries what English spreads across five words: "and they will write it." Frequency-based tokenizers rarely keep such inflected forms whole, so they shred them.
• Vocabulary is allocated by frequency — and the training mix is mostly English. A subword vocabulary is a fixed budget of a few tens or hundreds of thousands of entries, awarded to the character sequences most frequent in the tokenizer's training corpus. When that corpus is dominated by English and Latin script, Arabic gets a thin slice of the budget and its words fragment into short, low-information pieces.
• Byte fallback doubles the floor. When a sequence isn't in the vocabulary at all, byte-level tokenizers fall back to raw bytes — and every Arabic character is two bytes in UTF-8. The worst case for a rare Arabic word is not one token per letter; it is two. Orthographic variation (with and without diacritics, hamza and alef variants) multiplies the rare-sequence problem further.

Measuring the tax

Fertility is easy to measure and brutally clarifying: tokenize a representative sample of your own Arabic traffic and divide tokens by words. Published measurements and our own client audits consistently show the same pattern across tokenizer generations — dramatic improvement over time, but a persistent premium over English on general-purpose vocabularies:

What the tax actually costs you

Fertility is not an academic number. It multiplies through every token-denominated resource in your stack:

• Spend. At fertility 2.4 vs 1.3, the same support conversation costs ~1.8× more tokens in Arabic than English — before any difference in verbosity. Across millions of daily interactions, tokenizer choice is a line item.
• Latency. Decoding is per-token. An answer that fragments into 1.8× the tokens takes materially longer to stream — a user-visible penalty that no amount of prompt engineering removes.
• Context. A 128K-token window is not 128K tokens of content. High fertility shrinks how many Arabic words fit — squeezing RAG budgets, few-shot examples and long documents.

Put in procurement terms, the same workload prices very differently depending on the tokenizer underneath:

This is why comparing providers on price per million tokens is a category error for Arabic workloads. A model that looks 30% cheaper per token can be more expensive per conversation if its tokenizer fragments Arabic more aggressively. The only fair unit is cost per word — measured on your own traffic.

The quality connection

The tax is not only economic. Research on multilingual models has repeatedly found that high fragmentation is associated with weaker downstream performance: when a model must reassemble meaning from sub-morphemic shards, it spends capacity on spelling that better-tokenized languages spend on semantics. Fragmentation also degrades the plumbing around the model — embedding quality for retrieval, token-budget chunking that splits Arabic sentences mid-word, deduplication and caching that treat diacritized and undiacritized variants of the same text as unrelated strings, and exact-match evaluation metrics that punish orthographic variation the writer never controlled.

And the tax is regressive in exactly the wrong place: fertility is worst for the Arabic that matters most in production — dialectal, code-switched, informal text, which is rarest in tokenizer training corpora. Your cleanest MSA benchmark underestimates what your actual traffic will cost.

What to do about it

1. Measure fertility on your corpus, not a vendor's demo. Tokenize a real sample of your traffic — including dialect and code-switching — and compute tokens per word for every model on your shortlist. It is an afternoon of work that reprices the entire comparison.
2. Negotiate and forecast in cost per word. Convert every per-token price into per-word cost using your measured fertility before comparing vendors or projecting spend.
3. Budget context in words. Set RAG chunk sizes, few-shot counts and output limits from word budgets divided by measured fertility — not from token numbers borrowed from English deployments.
4. Prefer Arabic-aware vocabularies where you control the stack. If you are self-hosting or fine-tuning, tokenizer choice is a first-class architecture decision with a measurable payback in cost, latency and quality.
5. Normalize deliberately. Decide how you handle diacritics, hamza and alef variants at ingestion — consistently across training data, retrieval indexes and evaluation — so surface variation stops multiplying tokens and splitting caches.
6. Include fertility in evaluation. When you benchmark Arabic models, report tokens per word alongside quality scores. Two models with equal accuracy and unequal fertility are not equal products.