CoolFace
Modelpublic

optiviseapp/arabic-doc-extractor-qwen25vl-3b

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes
Model Card

Arabic Document Extractor โ€” Qwen2.5-VL-3B + QLoRA

๐Ÿญ Purpose: Extract structured data from Arabic PDF work orders, invoices, tables, and documents for factory automation.

Model Details

AttributeValue
Base ModelQwen/Qwen2.5-VL-3B-Instruct
MethodQLoRA (4-bit NF4) SFT via TRL
LoRArank=16, alpha=32, all-linear (vision + language)
Training RecipeBased on QARI-OCR โ€” SOTA Arabic OCR
Hyperparamslr=2e-4, batch=8 (eff.), 2 epochs, linear schedule, AdamW

Training Data

DatasetSamplesTask
Misraj/Misraj-DocOCR~thousandsArabic document โ†’ Markdown
Misraj/KITAB_pdf_to_markdown_reviewed~hundredsExpert-reviewed PDF โ†’ Markdown
ahmedheakl/arocrbench_tables~hundredsArabic tables โ†’ structured JSON

Capabilities

โœ… Arabic OCR โ€” Read printed Arabic text from scanned documents โœ… Structured Extraction โ€” Extract key-value pairs as JSON from work orders โœ… Table Extraction โ€” Convert Arabic financial/data tables to structured JSON โœ… Markdown Conversion โ€” Convert Arabic PDFs to formatted Markdown โœ… Bilingual โ€” Handles mixed Arabic/English documents

Quick Start

Installation

bash
pip install transformers peft torch qwen-vl-utils Pillow

Inference

python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
from PIL import Image
import torch

# Load base + adapter
base = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-3B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "optiviseapp/arabic-doc-extractor-qwen25vl-3b")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

# Extract from work order
image = Image.open("work_order.png").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "ุงุณุชุฎุฑุฌ ุฌู…ูŠุน ุงู„ุจูŠุงู†ุงุช ู…ู† ุฃู…ุฑ ุงู„ุนู…ู„ ู‡ุฐุง ุจุตูŠุบุฉ JSON"}
    ],
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
from qwen_vl_utils import process_vision_info
image_inputs, _ = process_vision_info(messages)
inputs = processor(text=[text], images=image_inputs, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=2000)
result = processor.batch_decode(
    [o[len(i):] for i, o in zip(inputs.input_ids, output)],
    skip_special_tokens=True
)[0]
print(result)

Work Order Extraction Prompt (Arabic)

ุงุณุชุฎุฑุฌ ุฌู…ูŠุน ุงู„ู…ุนู„ูˆู…ุงุช ู…ู† ู‡ุฐู‡ ุงู„ูˆุซูŠู‚ุฉ ุจุตูŠุบุฉ JSON ู…ู†ุธู…ุฉ ุชุดู…ู„:
- ุฑู‚ู…_ุงู„ุฃู…ุฑุŒ ุงู„ุชุงุฑูŠุฎุŒ ุงู„ู‚ุณู…ุŒ ุงู„ูˆุฑุฏูŠุฉ
- ุงุณู…_ุงู„ุนุงู…ู„ุŒ ุงู„ู…ู‡ู…ุฉุŒ ุงู„ุฃูˆู„ูˆูŠุฉุŒ ุงู„ุญุงู„ุฉ

Training

Run Training

bash
pip install transformers trl torch datasets trackio accelerate peft bitsandbytes qwen-vl-utils

# Set your HF token
export HF_TOKEN=your_token_here

# Run training (needs 24GB+ GPU โ€” A10G, A6000, or A100)
python train.py

Via HF Jobs

bash
huggingface-cli jobs run train.py \
  --hardware a10g-large \
  --timeout 6h \
  --dependencies transformers trl torch datasets trackio accelerate peft bitsandbytes qwen-vl-utils

Hardware Requirements

StageGPU VRAMRecommended
Training (QLoRA)16-24 GBA10G, A6000, RTX 4090
Inference (4-bit)6-8 GBRTX 3060+, T4
Inference (bf16)12-16 GBA10G, RTX 4090

๐Ÿ—๏ธ Factory Integration

For your factory automation platform:

  1. 1.PDF Upload โ†’ Convert pages to images (pdf2image library)
  2. 2.Extract โ†’ Run this model on each page with work order prompt
  3. 3.Parse JSON โ†’ Feed structured data to your shift assignment system
  4. 4.Assign โ†’ Auto-assign shifts based on extracted work order fields
python
from pdf2image import convert_from_path

# Convert uploaded PDF
pages = convert_from_path("uploaded_work_order.pdf", dpi=200)

# Extract from each page
for page in pages:
    result = extract_from_image(model, processor, page, task="work_order")
    work_order_data = json.loads(result)
    # Feed to your shift assignment system
    assign_shifts(work_order_data)

Improving Results

For best results on YOUR specific work orders:

  1. 1.Collect 100-500 annotated examples of your actual work orders with JSON ground truth
  2. 2.Add them to the training data and re-run fine-tuning
  3. 3.Use the QARI synthetic pipeline: Render your work order HTML templates โ†’ PDF โ†’ images with Arabic text variations

Related Models & References

ModelCERWERNotes
QARI-OCR v0.20.0610.160SOTA open-source Arabic OCR
AIN-7Bโ€”0.28Best Arabic multimodal (7B)
Baseerโ€”0.25Best doc-to-markdown
This modelTBDTBDSpecialized for structured extraction