Modelpublic

Nzyoka19/nyansapo-ocr-trocr-lora_v1

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes12downloads

Model Card

Nyansapo OCR — TrOCR LoRA v1

Fine-tuned OCR model by Nyansapo built on microsoft/trocr-base-handwritten with LoRA adapters on the decoder.

Training details

—Base model: microsoft/trocr-base-handwritten
—PEFT: LoRA (r=16, alpha=32)
—Targets: qproj, kproj, vproj, outproj, fc1, fc2
—Dataset: Nzyoka19/ocr_images

Inference

python

from transformers import VisionEncoderDecoderModel, TrOCRProcessor
from peft import PeftModel
from PIL import Image

base         = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-base-handwritten')
processor    = TrOCRProcessor.from_pretrained('Nzyoka19/nyansapo-ocr-trocr-lora_v1')
base.decoder = PeftModel.from_pretrained(base.decoder, 'Nzyoka19/nyansapo-ocr-trocr-lora_v1')
base.eval()

image         = Image.open('image.png').convert('RGB')
pixel_values  = processor(images=image, return_tensors='pt').pixel_values
ids           = base.generate(pixel_values)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])