Arabic Instruction-Tuning Data: Native vs Translated SFT Data

Good Arabic instruction tuning data is written, or at least rewritten, by native speakers for the varieties, domains and tasks your users actually bring. Translating English SFT sets teaches format cheaply but imports English syntax, Western content and an MSA-only register, and a controlled multilingual study found native instruction data pulling ahead on benchmarks built natively in the target language. The practical answer is a mix: translated scaffolding where it is harmless, native-authored data wherever dialect, culture or regional knowledge matters, and a written spec that says which is which.

What makes Arabic SFT data good?

SFT data pairs prompts with ideal responses and teaches behavior: format, register, length, refusals. For Arabic, "good" has five parts:

• Native phrasing. Prompts read like something an Arabic speaker would type; responses read like published Arabic, not a translation.
• The right register. User turns cover the varieties your users write in; responses follow a stated register policy.
• Regional knowledge. Names, places, currencies, institutions, calendars and cultural references belong to the region served.
• Arabic-specific tasks. Grammar, diacritization, dialect-to-MSA rewriting and Arabic document work.
• Provenance. Every item is tagged native, translated-and-localized, or generated-and-reviewed.

What does translated Arabic instruction data look like?

Translationese is grammatical text visibly shaped by its source language. It shows up from syntax to topic choice. The examples below were written for this article to illustrate each pattern; the first follows a correction described in the CIDAR paper.

A reviewer can fix most rows. Two need new writing: no edit turns translated MSA into a real Saudi user turn, and a Thanksgiving recipe localized into Kabsa is new content. That distinction drives the spec below.

Native vs translated Arabic instruction tuning data: what does the research say?

Controlled comparisons are scarcer than blog posts suggest, and the best one is not about Arabic:

The most useful result in Chen et al. is a round-trip experiment. Native Chinese data translated into English and back, which adds translation errors but keeps Chinese content, beat data translated from English on CMMLU for every base model tested. The authors read this as evidence that native knowledge matters: the cost of translated SFT data seems to come less from awkward phrasing than from content that was never about the target language's world.

The second lesson is about measurement: MGSM and MT-MMLU, both translated, could not tell the two kinds of training data apart. Score translated-tuned Arabic models on translated Arabic benchmarks and translation will look fine. Test on prompts written natively in Arabic, as in our guide to Arabic LLM evaluation.

Translation still dominates in practice. The Jais paper marks about 2.2 million of its 3.68 million Arabic instruction examples as translated; NativeQA-Ar, the set it wrote natively in Arabic for regional knowledge, has 15,018. A 2025 review of 366 Arabic post-training datasets on the Hugging Face Hub found cultural alignment in less than 1% of them and recommends native content, treating translation as a temporary bridge. Translation pipelines are improving: SmolKalam (a preprint marked work in progress, as of September 2026) translates SmolTalk2 with a model ensemble and, among other steps, filters on Arabic-content share and script purity. Such filters catch broken output. They cannot flag a Thanksgiving recipe in flawless Arabic.

Which dialects and registers should Arabic SFT data cover?

Separate the two sides of the conversation. User turns should look like your real traffic: the dialects your users type, MSA where they use it, Arabizi (Arabic in Latin letters and digits, such as 7abibi for حبيبي) and code-switching such as ابغى أحجز meeting بكرة ("I want to book a meeting tomorrow", Gulf). Assistant turns follow a written register policy.

Derive the user-side mix from logs or tickets, at the level your users differ: "Gulf" is often too coarse for a Saudi product. Our piece on MSA vs dialectal Arabic explains why missing varieties cap the model. Each dialect cell needs writers native to that variety; a Levantine writer imitating Najdi produces a third, fictional dialect.

Do not let a model verify dialect labels. In LabelBench, our reproducible audit (v0.1), a local language model reached 22.4% exact-match accuracy on five dialect regions on the fixed held-out manifest, where chance is 20%.

What domain and task mix does an Arabic instruction dataset need?

Start from your product's tasks, then add Arabic-only cells. In the 2025 review, machine-translation and question-answering datasets made up about 80% of those surveyed (42.3% and 38.3%), with none found for function calling or code generation.

• Open questions with regional knowledge: government services, regulation, geography, history, food.
• Rewriting and register conversion: dialect to MSA, informal to formal, simplification.
• Arabic document work: summarizing and extracting from contracts, invoices and circulars, with Hijri dates and Eastern Arabic digits.
• Language tasks: grammar (إعراب, i'rab), diacritization, spelling correction. CIDAR added grammar Q&A for this reason.
• Domain cells: medical, legal, financial and religious content, reviewed by licensed professionals.
• Multi-turn and safety: follow-ups that switch dialect mid-conversation, and natively phrased refusals.
• Tools and code: Arabic prompts that call tools or produce code, if your product does either.

A reusable Arabic SFT data spec

Fill this in before anyone writes or translates an item. The example column is illustrative, for a hypothetical Saudi banking assistant, not a recommended mix.

Measure length in tokens too: on an English-first tokenizer the same Arabic answer can cost far more tokens, which affects truncation and context budgets (see the Arabic tokenization tax). The Jais team excluded word-counting tasks from translation because word counts do not survive it. Reuse the style guide for Arabic preference data, so SFT and RLHF share one definition of a good answer.

How to QA Arabic SFT data before training

1. Provenance check. Every item is tagged; every cell respects its quota.
2. Variety check by native reviewers. A reviewer native to the variety confirms each item is that dialect, not MSA with a few colloquial words.
3. Translationese audit. On a sample, count Western versus regional names and places with Arabic NER, as the CIDAR team did, and flag English-shaped syntax.
4. Fact and domain review. Regional facts checked against sources; regulated-domain answers signed off by a qualified reviewer.
5. Format and script. Chat template, mixed-direction text, Arabic punctuation, digit policy, encoding. Language-ratio and script-purity filters are a cheap first pass.
6. Normalized deduplication. Normalize alef forms, final ya, ta marbuta, tatweel and diacritics first, or near-duplicates slip through.
7. Contamination check. Remove overlap with your hold-out set and any benchmark you report.
8. Gold set and adjudication. Score reviewers against gold items and adjudicate disagreements. An LLM judge can triage but should not be the only reviewer; our piece on LLM-as-a-judge in Arabic covers where automated graders fail.
9. Pilot ablation. Fine-tune on a pilot slice and score each cell on the native hold-out set before scaling.

This is how Bayanat Labs builds Arabic LLM training data: vetted native speakers and licensed domain professionals, screened for dialect and domain, write the dialect, cultural and domain cells in Arabic from the start. Work is calibrated against gold standards with adjudication and rework loops, with an audit trail on every task. The same style guide carries into Arabic RLHF and preference data, so fine-tuning and alignment reward the same kind of answer.