FatimahEmadEldin/Qari-OCR-Fine-Tuned-Kitab-Benchmark

Qari-OCR-Fine-Tuned-Kitab-Benchmark

Model Description

This model is a LoRA fine-tuned version of NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct specifically optimized for Arabic OCR tasks using the comprehensive KITAB-Bench dataset.

Model Details

• —Base Model: NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct
• —Model Type: Vision-Language Model with LoRA fine-tuning
• —Language: Arabic (primary), with multilingual capabilities
• —License: [Specify license]
• —Fine-tuned for: Arabic Optical Character Recognition (OCR)

Training Configuration

• —Training Method: LoRA (Low-Rank Adaptation)
• —LoRA Parameters:
• —Rank (r): 16
• —Alpha: 32
• —Dropout: 0.05
• —Target Modules: ["qproj", "kproj", "vproj", "oproj", "gateproj", "upproj", "down_proj"]
• —Training Epochs: 5
• —Batch Size: 4 per device
• —Learning Rate: 2e-4
• —Optimizer: AdamW 8-bit
• —Max Sequence Length: 2048

Dataset

The model was trained on a curated subset of Arabic OCR datasets comprising 3,760 total samples from 13 domain-specific datasets:

Training Data Composition

Total Combined Dataset: 3,760 samples

• —Training Set: 3,572 samples (95% of total)
• —Held-out Test Set: 188 samples (5% of total)

Source Datasets Used:

• —ahmedheakl/arocrbench_synthesizear: 500 samples
• —ahmedheakl/arocrbench_patsocr: 500 samples
• —ahmedheakl/arocrbench_historyar: 200 samples
• —ahmedheakl/arocrbench_historicalbooks: 10 samples
• —ahmedheakl/arocrbench_khattparagraph: 200 samples
• —ahmedheakl/arocrbench_adab: 200 samples
• —ahmedheakl/arocrbench_muharaf: 200 samples
• —ahmedheakl/arocrbench_onlinekhatt: 200 samples
• —ahmedheakl/arocrbench_khatt: 200 samples
• —ahmedheakl/arocrbench_isippt: 500 samples
• —ahmedheakl/arocrbench_arabicocr: 50 samples
• —ahmedheakl/arocrbench_hindawi: 200 samples
• —ahmedheakl/arocrbench_evarest: 800 samples

Data Split

• —95% training (3,572 samples)
• —5% held-out test (188 samples) for final evaluation

Domain Coverage

• —Handwritten Text: Historical manuscripts, personal notes, traditional calligraphy
• —Printed Text: Books, newspapers, academic papers, legal documents
• —Scene Text: Street signs, advertisements, natural environments
• —Structured Documents: Tables, forms, layouts
• —Historical Documents: Ancient texts, heritage manuscripts
• —Synthetic Data: Generated text for augmentation

Performance

Evaluation Results on Held-Out Test Set

Intended Use

Primary Use Cases

• —Arabic document digitization
• —Historical manuscript transcription
• —Multi-domain Arabic text recognition
• —RAG (Retrieval-Augmented Generation) document processing pipelines
• —Academic research in Arabic NLP and OCR

Direct Use

from transformers import AutoProcessor, AutoModelForVision2Seq
from PIL import Image

# Load model and processor
processor = AutoProcessor.from_pretrained("FatimahEmadEldin/Qari-OCR-Fine-Tuned-Kitab-Benchmark")
model = AutoModelForVision2Seq.from_pretrained("FatimahEmadEldin/Qari-OCR-Fine-Tuned-Kitab-Benchmark")

# Process image
image = Image.open("arabic_document.jpg")
prompt = "Below is the image of one page of a document. Please provide the plain text representation of this document as if you were reading it naturally, ensuring high accuracy."

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": prompt},
        ],
    }
]

text_prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text_prompt, images=image, return_tensors="pt")

# Generate
generated_ids = model.generate(**inputs, max_new_tokens=2048)
generated_ids = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Limitations and Considerations

Known Limitations

• —Complex Fonts: Performance may vary with highly stylized or decorative Arabic fonts
• —Numeral Recognition: Some challenges with mixed Arabic-Indic numeral systems
• —Word Elongation: Handling of kashida (Arabic text elongation) requires improvement
• —PDF-to-Markdown: Limited accuracy (best models achieve ~65% on complex layouts)

Bias and Fairness

• —Trained primarily on Modern Standard Arabic; dialectical variations may have reduced accuracy
• —Historical document performance depends on manuscript quality and preservation state
• —Geographic bias toward Gulf and Levantine Arabic text styles

Technical Specifications

Hardware Requirements

• —Minimum: 8GB GPU memory for inference
• —Recommended: 16GB+ GPU memory for optimal performance
• —Training: Conducted on NVIDIA A100 GPUs

Software Dependencies

• —transformers >= 4.51.3
• —torch >= 2.4.0
• —unsloth (for efficient training)
• —Pillow for image processing

Training Details

Training Infrastructure

• —Framework: Unsloth for efficient LoRA training
• —Quantization: 4-bit quantization for memory efficiency
• —Mixed Precision: BF16/FP16 based on hardware support

Data Processing

• —Images processed at various resolutions maintaining aspect ratios
• —Text preprocessing includes normalization of Arabic diacritics
• —Synthetic data generation pipeline for augmentation

Citation

If you use this model, cite the original dataset:

@article{heakl2025kitab,
  title={KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding},
  author={Heakl, Ahmed and Sohail, Abdullah and Ranjan, Mukul and Hossam, Rania and Ahmad, Ghazi and El-Geish, Mohamed and Maher, Omar and Shen, Zhiqiang and Khan, Fahad and Khan, Salman},
  journal={arXiv preprint arXiv:2502.14949},
  year={2025}
}

Related Models

• —Base Model: NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct
• —KITAB-Bench Collection: ahmedheakl/kitab-bench