mabdulaziz499/arabic-htr-fanar-7b-lora

arabic-htr-fanar-7b-lora

Arabic handwritten text recognition (HTR). A QLoRA adapter for `QCRI/Fanar-2-Oryx-IVU` that turns a handwritten Arabic line image into Arabic Unicode text.

On the held-out KHATT v1.0 test split (1038 real handwritten lines) it reaches 3.77% CER / 16.50% WER — 36% fewer character errors than the best other model measured here (Arabic-English-handwritten-OCR-v3, 5.86% CER, zero-shot). Fine-tuning took the same base model from 50.98% → 3.77% CER (93% error reduction).

Read this before comparing: this adapter is fine-tuned on the KHATT training split and evaluated on the KHATT test split — an in-domain result. The other models in the table below are evaluated zero-shot; they were not trained on KHATT. The table therefore shows "how well does each model read KHATT handwriting today", not "which model is better in general". A model fine-tuned on their data would score differently. Every number was measured under one identical protocol (same images, prompt, greedy decoding, normalization) — see Evaluation.

Results

CER = character error rate (verbatim: diacritics and punctuation count). Lower is better. Full raw numbers: `benchmark.json` in this repo.

[image]

Corpus-level CER hides the shape of the errors, so here is the quality profile — what share of lines land under a given error threshold:

[image]

[image]

Quality profile of this model: median line CER 1.89%, with 77.6% of lines at ≤5% CER, 92.5% at ≤10%, and 29.4% transcribed exactly.

Sample predictions

Usage

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
from PIL import Image

BASE = "QCRI/Fanar-2-Oryx-IVU"
ADAPTER = "mabdulaziz499/arabic-htr-fanar-7b-lora"

processor = AutoProcessor.from_pretrained(BASE, min_pixels=200704, max_pixels=802816)
model = AutoModelForImageTextToText.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

PROMPT = "Transcribe the handwritten Arabic text in this image exactly as written, including any diacritics. Output only the transcription, nothing else."
image = Image.open("line.png").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": PROMPT}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=160, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0].strip())

Input expectations: one single line of handwritten Arabic per image (as in KHATT). For a paragraph or a form, segment it into lines first and call the model per line — see Limitations.

Output: Arabic text in logical Unicode order (NFC). Use arabic-reshaper + python-bidi only for terminal display, never for storage or scoring.

Training

Data hygiene: KHATT contains fixed paragraphs copied verbatim by many writers, so the same sentence appears in both train and test in different handwriting. Every training line whose exact text occurs in the test split was removed before training (~1,700 lines). Without that filter a model can memorize the test text and report a flattering CER.

Evaluation

All models in the table were scored by the same script under one protocol:

• —Data: local_khatt:test — 1038 real handwritten lines, held out.
• —Prompt: identical for every model (above).
• —Decoding: greedy (dosample=False), `maxnew_tokens=160`.
• —Image budget: identical (min_pixels=200704, max_pixels=802816).
• —Metric normalization (NFC, logical order, whitespace-collapsed, diacritics KEPT): references and hypotheses pass through the same normalizer; diacritics are kept and count as errors (verbatim scoring). A diacritic-insensitive secondary score is in benchmark.json.
• —Text order: KHATT mirrors store ground truth in visual (reversed) order; it is restored to logical order by grapheme-cluster reversal before scoring. Getting this wrong silently invalidates CER.

Limitations

• —Single-line recognizer. Multi-line images are out of distribution: the model may fall back to a memorized KHATT sentence. Segment into lines first.
• —Domain. Trained on KHATT: mostly young Saudi writers, clean scans, black ink on white paper, MSA prose. Photos, ruled/form paper, stamps, cheque-style amounts, dialectal or heavily abbreviated writing are all further from the training distribution — expect degradation.
• —In-domain result. 3.77% CER is a KHATT number. Your handwriting is not KHATT; measure on your own data before trusting it.
• —Not printed-text OCR. For printed Arabic use a print OCR model.
• —Numbers/dates are rare in KHATT and correspondingly weak.
• —No diacritic supervision. KHATT labels are largely undiacritized, so the model outputs bare text.

License & data terms

The adapter is released under Apache-2.0, matching the base model `QCRI/Fanar-2-Oryx-IVU` (Apache-2.0). Using it requires downloading that base model under its own license. It was trained on KHATT v1.0, distributed for research; check the dataset's terms before commercial use and cite the KHATT authors in publications.

Citation

@misc{arabic_htr_fanar_7b_lora,
  title  = {arabic-htr-fanar-7b-lora: a QLoRA adapter for handwritten Arabic line recognition},
  author = {Abdelaziz, Mahmoud},
  year   = {2026},
  url    = {https://huggingface.co/mabdulaziz499/arabic-htr-fanar-7b-lora}
}

Please also cite KHATT (Mahmoud et al., Pattern Recognition, 2014) and the base model Fanar-2-Oryx-IVU.