Modelpublic

vaghawan/gemma-4-31b-it-merged-16bit-checkpoint-11700

sourceHugging Facegemmaupdated 27d agoView on Hugging Face

0likes18downloads

Model Card

gemma-4-31b-it-merged-16bit-checkpoint-11700

Merged 16-bit (bfloat16) checkpoint of unsloth/gemma-4-31B-it with LoRA adapters from training step 11700.

LoRA is baked into the base weights (merge_and_unload). This is a full Gemma4ForConditionalGeneration checkpoint and can be served with vLLM without --enable-lora.

Source adapters: /workspace/llm-finetune/checkpoint/checkpoint-11700.

Validation loss (step 11700)

Metric	Value
`eval_all_loss`	0.831619
`eval_anchor_loss`	0.803829
`eval_english_chat_loss`	1.123160
`eval_hausa_chat_loss`	1.198374
`eval_tool_loss`	0.210816

Serve with vLLM

Requires a vLLM build that includes Gemma 4 (vLLM ≥ 0.19). 31B bf16 needs about 62 GB of VRAM (one 80 GB GPU, or tensor parallel across two).

bash

vllm serve vaghawan/gemma-4-31b-it-merged-16bit-checkpoint-11700 \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4

Multi-GPU:

bash

vllm serve vaghawan/gemma-4-31b-it-merged-16bit-checkpoint-11700 \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4

Load with Transformers

python

from transformers import AutoProcessor, Gemma4ForConditionalGeneration

model = Gemma4ForConditionalGeneration.from_pretrained(
    "vaghawan/gemma-4-31b-it-merged-16bit-checkpoint-11700",
    torch_dtype="bfloat16",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("vaghawan/gemma-4-31b-it-merged-16bit-checkpoint-11700")