Kimang18/paddleOCR_vl_Khmer_finetuned
Objective
Fine-tune PaddlePaddle/PaddleOCR-VL on `WildKhmerST`.
For future benchmarking in Khmer OCR task, `WildKhmerST` is processed to contains pairs of 'text region image' and 'text'. The processed dataset is uploaded to huggingface platform here. There are 3 splits. train split is used to full fine-tune PaddlePaddle/PaddleOCR-VL. test split is used to evaluate the fine-tuned model.
Uploaded model
- Developed by: Kimang18
- License: apache-2.0
- Finetuned from model : paddleocrft
This paddleocr_vl model was trained 2x faster with Unsloth
Usage
Inference using Huggingface transformers on NVIDIA GPUs.
Requirements tested on python 3.12.13, T4 GPU:
transformers<5.0.0from PIL import Image
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
# ---- Settings ----
model_path = "Kimang18/paddleOCR_vl_Khmer_finetuned"
image_path = "khmer_text_region.png"
# ------------------
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
image = Image.open(image_path).convert("RGB")
model = AutoModelForCausalLM.from_pretrained(
model_path, trust_remote_code=True, dtype=torch.bfloat16
).to(DEVICE).eval()
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
messages = [
{"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "OCR:"},
]
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt"
).to(DEVICE)
outputs = model.generate(**inputs, max_new_tokens=1024)
outputs = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(outputs)_NOTE: if you use `transformers>=5.0.0`, you will encounter `KeyError` from `ropeinitfn=ROPEINITFUNCTIONS[ropetype]. This is because PaddleOCR-VL uses rope_type="default" which is not supported anymore for transformers>=5.0.0`.