shubham879/trocr-prescription

Model Card for trocr-prescription

Model Details

Model Description

This model is a fine-tuned version of microsoft/trocr-base-handwritten for handwritten medical prescription text recognition.

It is designed to extract handwritten prescription text from medical images using transformer-based OCR.

• —Developed by: shubham879
• —Model type: VisionEncoderDecoderModel
• —Language(s): English
• —License: Apache-2.0
• —Finetuned from model: microsoft/trocr-base-handwritten

Model Sources

• —Repository: https://huggingface.co/shubham879/trocr-prescription
• —Demo: https://prescription-error-detection.vercel.app/

Uses

Direct Use

This model is intended for:

• —handwritten prescription OCR
• —medical text extraction
• —OCR preprocessing pipelines

Downstream Use

Can be integrated into:

• —prescription error detection systems
• —medical document digitization
• —healthcare OCR pipelines

Out-of-Scope Use

Not recommended for:

• —printed document OCR
• —multilingual handwriting
• —severely distorted or noisy scans

Bias, Risks, and Limitations

• —Accuracy decreases on unclear handwriting
• —Sensitive to poor image quality
• —Drug names with unusual handwriting may be misread

Recommendations

Use cropped line-level prescription images for best performance.

How to Get Started with the Model

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

processor = TrOCRProcessor.from_pretrained("shubham879/trocr-prescription")
model = VisionEncoderDecoderModel.from_pretrained("shubham879/trocr-prescription")

image = Image.open("sample.png").convert("RGB")

pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)

text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(text)

Training Details

Training Data

The model was fine-tuned using the Doctor’s Handwritten Prescription BD dataset, which contains handwritten medical prescription images.

Training Procedure

Preprocessing

• —Images resized using TrOCRProcessor
• —RGB conversion applied
• —Tokenization handled by TrOCRProcessor

Training Hyperparameters

• —Training regime: fp32

Evaluation

Metrics

• —Character Error Rate (CER)
• —Sequence Accuracy

Results

• —CER: 9%
• —Accuracy: 73%

The model performs well on handwritten prescription text at line level with low character-level error.