Arabic token-tax calculator: tokens, context and cost for your text

The Arabic Token-Tax Calculator by Bayanat Labs shows what your own Arabic text takes in tokens. Paste Arabic text, and its English translation if you have one, and see for each tokenizer the token count, tokens per word, the Arabic/English token ratio, characters per token, whether the text fits a context budget, and the cost of sending or generating it at list prices read from the vendors' price pages on 30 October 2026, which you can edit. The calculator runs in your browser with 9 tokenizers; the command-line tool token-tax runs 12 and gives the same token ids.

Open the calculator · Code on GitHub · Release record

For the gap measured on parallel UN text, tokenizer by tokenizer, see the Arabic token counter and the Arabic Token-Tax Index: across its 12 tokenizers, the median Arabic/English token ratio is 1.42.

In the browser

• Tokens for each tokenizer, encoded exactly as you wrote the text, with no special tokens.
• Tokens per word and per thousand words. Arabic clitics stay attached to their words, so these overstate the gap between the languages.
• The Arabic/English token ratio, when you add the English parallel: Arabic tokens ÷ English tokens on the same tokenizer. This compares the same content and is the number to quote.
• Characters per token.
• Context budget: whether the text fits, by how many tokens it is over and the share that fits.
• Cost of sending (input price) and generating (output price) a number of texts like yours, at 7 list prices from OpenAI, DeepSeek and Alibaba Cloud read on 30 October 2026; one of them is a promotional price and is marked as such. Edit any price or add your own.
• A share card: a PNG with the headline ratio, the tokenizer revisions and the date, drawn in your browser. Your text is left out unless you choose to include its opening characters.

The interface is in English and Arabic.

Your text stays in your browser

The tokenizers run in a Web Worker on the page, from tokenizer files the page serves itself and checks by SHA-256. The page's Content-Security-Policy allows network connections to its own origin only, and the page stores nothing. A check in Chromium measured a long text and recorded 0 requests outside the page's origin and nothing written to browser storage. On 242 texts and 11 edge-case strings, the page gives the same token ids as the Python tool for each of its 9 tokenizers.

On the command line

pip install "git+https://github.com/Bayanat-Labs/arabic-token-tax-calculator@v0.1.0"

token-tax count "قالت البلدية إن أعمال صيانة الطريق الساحلي ستستمر ثلاثة أسابيع" \
  --en "The municipality said that maintenance work on the coastal road will continue for three weeks"
token-tax count --file ar.txt --en-file en.txt --tokenizers o200k_base,qwen3.5 --budget 8192
token-tax count --file ar.txt --json        # all measures and costs as JSON
token-tax tokenizers                        # tokenizers, revisions, licences
token-tax prices                            # the dated list prices

Python 3.10 or later. The package is not on PyPI yet; install it from GitHub as above. Tokenizer files are downloaded from their model repositories on first use, at pinned revisions and checked by SHA-256: tokenizer files only, no model weights and no model code.

Tokenizers

• In the browser and on the command line: OpenAI o200k_base, OpenAI cl100k_base, Qwen3, Qwen3.5 / 3.6 / 3.8, DeepSeek V3 / V4, Mistral Tekken, Gemma 4 / Fanar-2, ALLaM and Fanar-1.
• On the command line only: Falcon3, Falcon-H1 and Falcon-H1R, whose tokenizer files are not under MIT or Apache-2.0.
• The list, the pinned revisions and the priced models come from the Arabic Token-Tax Index 0.1.0; the prices were read again from the vendors' official price pages on 30 October 2026.

Cite as

Bayanat Labs (2026). Arabic Token-Tax Calculator, version 0.1.0. https://github.com/Bayanat-Labs/arabic-token-tax-calculator

The release record, with licences and changelog, is the Arabic Token-Tax Calculator on Bayanat Labs Open.

Licence and contact

Code and calculator page: Apache-2.0. Fixtures, price table and documentation: CC BY 4.0. Tokenizer files in the calculator page keep their own licences (MIT or Apache-2.0). Questions, corrections and requests for new tokenizers: hello@bayanatlabs.com.

Need Arabic evaluation data for your models?

The calculator is free. For licensed Arabic evaluation sets and private test suites, see Bayanat Bench.