CoolFace
Modelpublic

topor4ik/qwen3vl-rukopys-band-dense-v1

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes7downloads
Model Card

LoRA adapter: Qwen3-VL-8B for Ukrainian handwriting (RUKOPYS)

Language-only QLoRA adapter (vision tower frozen) that turns Qwen3-VL-8B-Instruct into a single-call detector + transcriber for Ukrainian handwritten document pages — one VLM call returns a JSON list of {bbox, type, text} regions. Trained for the Kaggle Handwritten-to-Data competition on the RUKOPYS dataset.

Unmerged on purpose: serve it with vLLM --enable-lora on a shared Qwen/Qwen3-VL-8B-Instruct base — multiple adapter versions can run side-by-side on one base. The base weights are NOT in this repo; only the ~200 MB LoRA delta + adapter_config.json + processor (which are identical to the base under language-only LoRA).

Model details

  • —Fine-tuned from: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit (serve/merge on Qwen/Qwen3-VL-8B-Instruct)
  • —Method: QLoRA via Unsloth, language layers only (vision frozen)
  • —LoRA: r=32, alpha=64, dropout=0
  • —Task: document region detection + handwritten text recognition (HTR)
  • —Language: Ukrainian (uk)
  • —License: apache-2.0

Training

  • —Epochs: 2 (eval/early-stop on)
  • —Effective batch: 1 × 8 grad-accum = 8
  • —LR: 0.0002 (cosine, 50 warmup), seed 3407
  • —Image budget: ≤1024² px; max seq length: 4096
  • —Optimizer: adamw_8bit; precision: 4-bit NF4 base + bf16 LoRA

Usage (vLLM --enable-lora)

bash
python -m vllm.entrypoints.openai.api_server \
  --model Qwen/Qwen3-VL-8B-Instruct \
  --enable-lora --lora-modules qwen3vl-rukopys-band-dense-v1=topor4ik/qwen3vl-rukopys-band-dense-v1 \
  --max-lora-rank 32 --max-model-len 16384

Then send chat-completions with model=<lora-module-name> and an image; the model returns {"regions": [{"bbox":[x1,y1,x2,y2], "type":"...", "text":"..."}]} with bbox in 0–1000 scale, top-to-bottom reading order.