Research · Open kit v0.1

Arabic scene-text and visual QA inspection kit

Six synthetic images, 40 boxed text regions and 18 Arabic questions for checking whether a vision-language or OCR model reads Arabic in images correctly and answers from the right place. Each question targets a known way models can fail: Eastern Arabic digits, abbreviations, bilingual signs, rows on a menu, arrow direction, negation and text it cannot see.

Bayanat Labs Research· · Synthetic images · CC BY 4.0

A vision-language model can describe a street scene fluently and still misread the sign in it. With Arabic the errors are specific: a model can read the Eastern Arabic digit ٥ as a zero, drop the م that makes a closing time p.m., pair a menu item with the price on the row below, or mirror an arrow because the text around it runs right to left. Ask about something a sticker covers and it may answer anyway.

This kit is a short, repeatable test for those errors. Bayanat Labs drew six scenes with Saudi conventions (riyal prices, Eastern Arabic digits, Riyadh-area place names), marked every text region with a box and its transcription in reading order, and wrote 18 questions in Arabic, each tied to the regions that hold the answer. Two questions cannot be answered from the image, on purpose. Use it to compare models or versions before a launch, to check an OCR pipeline on Arabic text in scenes, and to agree an image-record format with your data supplier before a custom collection.

بالعربية: مجموعة مجانية ومفتوحة لفحص قدرة نماذج الرؤية واللغة وأنظمة التعرّف الضوئي على الحروف (OCR) على قراءة النص العربي في الصور والإجابة عن أسئلة مرتبطة بموضع الدليل في الصورة: ست صور اصطناعية (واجهة مخبز، ولوحة إرشادية للطريق، وعبوة حليب، وقائمة مشروبات، وملصق منظف أرضيات، ولوحة منع وقوف يغطي ملصقٌ جزءًا منها)، فيها 40 منطقة نصية بإحداثياتها ونصها بترتيب القراءة، و18 سؤالًا بالعربية لكل منها إجابة مرجعية ومناطق الدليل، منها سؤالان لا يمكن الإجابة عنهما من الصورة. وتضم المجموعة دليل فحص بقواعد لتوحيد الكتابة (الأرقام وأشكال الألف والتاء المربوطة) و16 رمزًا للأخطاء، ومخططًا لسجل الصورة يحدد نوع المصدر وظروف التصوير وطريقة التعامل مع الأشخاص ومجموعة التقسيم، ويفصل كل حق من حقوق الاستخدام عن غيره. الصور رسوم أعدّتها Bayanat Labs، وليست صورًا فوتوغرافية، ولا يظهر فيها أشخاص، والمنشآت فيها متخيَّلة. لبيانات الصور والفيديو المخصصة، راجع بيانات الصور والفيديو العربية.

What is in the kit

FileContents
images/SC01.png to SC06.pngSix 1200 × 800 PNG illustrations drawn by Bayanat Labs: a bakery storefront, a road direction sign, a milk carton, a cafe menu board, a floor-cleaner label and a parking notice partly covered by a sticker. No people, no real brands.
questions.jsonl18 Arabic questions with English glosses, reference answers, the evidence region IDs, an answerable flag and the error codes each question can catch. Two are unanswerable: one because the information is not in the image, one because it is covered.
annotations.jsonlOne record per image: 40 text regions (34 Arabic, 6 Latin) with a box, the text as written in reading order, a script tag and a digit tag (13 regions use Eastern Arabic digits), 4 non-text regions (three arrows and the sticker), the questions, capture conditions, source type and rights.
rubric.mdA six-step protocol, a default normalization policy for digits, alef forms and ta marbuta, 16 error codes for reading and answering, the grounding and abstention rules, and the record checks to agree with a supplier.
schema.jsonJSON Schema (draft 2020-12) for one annotated image: source type and a generated flag, capture conditions, people in frame, split group, text regions, grounded questions and a separate field for each right.
README.md, SHA256SUMS.txt, LICENSE.txtNotes, checksums for every file, and the CC BY 4.0 license.
Synthetic blue road sign with three rows: الرياض (Riyadh) 25 km with an up arrow, الخرج (Al Kharj) 80 km with a right arrow, and المطار (Airport) 12 km with a left arrow; distances in Eastern Arabic digits.

SC02, road sign. Three destinations, each with an Arabic name, an English name, a distance in Eastern Arabic digits and an arrow. Q06 asks which way the arrow to the airport points (left); a model that mirrors directions for right-to-left text says right.

Six scenes and what each tests

SceneWhat is in itQuestions and what they test
SC01 storefrontSign مخبز الحي with "Al Hay Bakery" below, a sticker خصم ٢٠٪, hours ٦ ص to ١١ مQ01 the Arabic name (dots, Arabic vs English); Q02 the discount (digits and the Arabic percent sign); Q03 closing time (م means p.m., and not the opening time); Q04 phone number, which is not in the image
SC02 road signRiyadh, Al Kharj and the airport, each with a distance and an arrowQ05 distance to Al Kharj (right row, unit); Q06 arrow direction; Q07 the closest destination (compare Eastern Arabic numbers)
SC03 milk cartonProduct name, ١ لتر, a production date and an expiry date one week apart, storage lineQ08 expiry date (not the production date); Q09 volume with its unit; Q10 storage
SC04 menu boardFour drinks in a right-hand column, prices in a left-hand column, "prices in Saudi riyals" footnoteQ11 and Q12 prices (the right row, the currency from the footnote); Q13 orange juice, which the menu does not list
SC05 product labelDilution instruction يخفف ٣٠ مل في ٥ لتر ماء and the warning لا يخلط مع الكلورQ14 the amount with its unit; Q15 whether it can be mixed with chlorine (negation)
SC06 parking noticeممنوع الوقوف, من ٨ ص إلى with the end time under a sticker, ما عدا يوم الجمعةQ16 start time; Q17 end time, which is covered, so the correct response is that it cannot be read; Q18 Friday (the exception)

How to inspect a model with it

  1. Write the reading policy first. Decide whether Eastern Arabic and Western digits count as equal, which letter forms you normalize and whether abbreviations must be expanded. rubric.md has a default; change it before you run, not after.
  2. Test reading and answering separately. First ask for all the text in each image, in reading order, and compare it with the transcriptions. Then ask the 18 questions one at a time, in Arabic, with the image.
  3. Keep the first output. Use the same prompt and settings for every model, and fix the temperature or seed, so two models or two versions are compared on equal terms.
  4. Check where the answer came from. If the model returns a box, compare it with the question's evidence regions (intersection over union of 0.5 or more). A right answer from the wrong place is luck.
  5. Report by error code, not one score. A model that fails every digit question needs different work from one that answers Q17 with a guessed time. Keep the failing outputs and run the same items after a fix.

Eighteen questions are a smoke test, not a benchmark. The images are clean illustrations; real photos add glare, blur, angle and handwriting. Before launch, add your own images and questions in the same format. Our 10-point audit for Arabic image and video datasets covers what to check in the data itself.

Synthetic no-parking notice reading ممنوع الوقوف, then من ٨ ص إلى with a yellow sticker covering the end time, then ما عدا يوم الجمعة and No Parking.

SC06, parking notice. The end time is drawn under the sticker and cannot be seen. Q17 asks for it; the correct response says it cannot be read.

A worked example: the answer the image does not give

Q17 asks حتى أي ساعة يمنع الوقوف؟ ("Until what time is parking banned?"). The notice reads من ٨ ص إلى and then a sticker. The failure to look for is a plausible guess such as 4 or 5 p.m.; the image cannot support either. The question is recorded as:

{"question_id": "Q17", "scene_id": "SC06",
 "question_ar": "حتى أي ساعة يمنع الوقوف؟", "answerable": false,
 "unanswerable_reason": "not_legible",
 "evidence_regions": ["SC06-R2", "SC06-O1"],
 "expected_response_ar": "لا يمكن معرفة ذلك، لأن الملصق يغطي وقت الانتهاء.",
 "tests": ["QA_NO_ABSTAIN", "TXT_HALLUC"]}

Q04 (the bakery's phone number) is the other kind: the information is simply not in the image (not_present). Q13, Q15 and Q18 are different again: their answer is "no", and the image shows why. Keeping these three cases apart is what tells you whether a model abstains for the right reason.

Agree the image record before you collect

The schema gives every image the fields a buyer needs to trust a dataset later. source.type is field capture, client-supplied or generated, and generated is a separate flag, so results on real photos can be reported on their own; every image in this kit is marked generated. capture_conditions records lighting, angle, distance, blur, glare and occlusion. people_in_frame states whether people appear and how they were handled; labels describe what is visible and never infer anyone's identity from appearance. split_group names the store, location or source, so all images from one place stay in one split. rights has one field each for model training, evaluation, redistribution and public display, each granted, not_granted or not_stated, with a pointer to the permission record. A right that is not written down is not_stated, never assumed.

What this kit is not

  • Not photographs. The six images are flat illustrations rendered to PNG, with even lighting and no blur. Passing them says little about real photos; failing them is a clear signal.
  • Not a sample of a Bayanat dataset. Bayanat Vision is built to order. Bayanat Labs drew these scenes for this kit; the businesses and products are fictional.
  • Not a benchmark. It has no scores and ranks no model. Eighteen questions show where a model fails; they do not measure how often.
  • Not every script style or region. v0.1 uses one typeface and Saudi conventions. Handwriting, calligraphic shop signs, Maghrebi conventions and video are not covered.

Build Arabic image and video data with Bayanat Vision

Bayanat Vision builds the real-photo version of this kit to your scope:

  • Labels that show where an answer comes from. Scene text, bounding boxes, object relationships, answers linked to visible evidence and, for video, event timestamps (Bayanat Vision label types).
  • A capture plan around your task. Lighting, capture conditions, text styles, layouts and difficulty planned for the signs, menus, packaging and forms your users meet (how to scope an image or video project).
  • Generated images kept apart. Generated images remain identifiable separately from field collection, as every image in this kit is (Bayanat Vision).
  • Permissions first, no identity guessing. Source and media permissions are agreed before collection, and the collection is described without inferring people's identities from appearance (use rights).
  • Arabic readers from a vetted team. A team of 50,000+ expert annotators, including vetted native speakers and linguists across 25+ Arabic varieties, who pass dialect and domain screening and are calibrated against gold standards (Arabic data annotation services).
  • A pilot first. A fixed-scope pilot with gold-standard QA and a quality report, usually within two weeks, with in-region hosting by default and private-cloud or on-premises options (start a pilot).

Cite or reuse

Free to use, adapt and redistribute under CC BY 4.0 with attribution:

Bayanat Labs (2026). Arabic Scene-Text and Visual QA Inspection Kit v0.1. https://www.bayanatlabs.com/research/arabic-scene-text-vqa-kit

Found an error, or want scenes for another region, script style or domain? Write to hello@bayanatlabs.com.

Scope an Arabic image or video dataset with Bayanat Vision

Tell us what your model has to read, see or point to, the scenes and capture conditions, and how the data will be used. We come back with a fixed-scope pilot plan and the permissions to agree.

Scope a Bayanat Vision dataset
Bayanat Labs © 2026 Bayanat Labs · Riyadh