Modelpublic

tzchen07/gemma4-31b-rovochat-full-sft-w1-260729

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes16downloads

Model Card

gemma4-31b-rovochat-full-sft-w1-260729

Full-parameter supervised fine-tune of Gemma-4-31B-it for multi-turn tool-calling / agent orchestration. Trained 2026-07-29 → 2026-07-31 on 16×H200 (2 nodes, GPUDirect-RDMA).

## ⚠️ This model has not been evaluated These weights are the raw output of a training run, published for reproducibility and further work. No benchmark, no quality gate, and no safety evaluation has been run on this checkpoint. Training loss converged cleanly — that is not evidence the model is good. In this model family, full-parameter fine-tuning is specifically known to degrade open-ended text quality while tool metrics improve, and the run's KL-to-base regularizer was disabled under full-FT (it requires an adapter). Treat text quality as unverified and plausibly regressed versus the base model.

Training


method	full-parameter FT (all non-vision params), DeepSpeed ZeRO-3, no offload, bf16
trainable	30.697 B / 31.273 B (98.16 %) — vision tower frozen
hardware	16 × H200 141 GB, 2 nodes, RoCE/RDMA interconnect
epochs	1 — 457 optimizer steps, global batch 16 (micro-batch 1 × 16 ranks, no accumulation)
optimizer	AdamW, LR 2e-6 cosine, warmup 3 %, weight decay 0.1, grad-clip 1.0
sequence	up to 44 k tokens (long multi-turn conversations, assistant-turn-masked loss)
attention	SDPA (memory-efficient) — no FlashAttention
seed	17
wall-clock	44 h 25 m, mean 350 s/step, no restarts
final loss	0.80 – 1.02 (from ~3.4 at init)
checkpointing	every 45 steps, keep newest 2; 12/12 saves succeeded (11 checkpoints + final)

Measured loss decomposition

Verified on all 457 steps: total == ce + 4·fd + 4·tc + 0.1·anchor (max deviation 9.4e-07). The KL anchor was exactly 0.0 on every step — it is inactive under full-parameter training, so nothing regularized this model toward the base distribution.

From rank-0 samples (one micro-batch per step of 16, so shares are estimates, not exact run-level values): the shaped terms fall from ~32 % of the objective over the first 50 steps to ~6 % over the last 50 (CE ~94 %, 4·fd ~4 %, 4·tc ~1 %) — the first-decision term is essentially solved during warmup.

Full-run statistics (all 16 ranks): loss minimum 0.690 @ step 199; block means 1–99 1.648 → 380–457 0.980. Pre-clip grad-norm peaked at 206,560 @ step 62, absorbed by max_grad_norm=1.0.

Loss

A shaped objective over assistant turns, combining:

1.Per-turn cross-entropy — mean within each turn, then mean across turns, so long turns don't dominate the gradient (a fix for final-turn concentration in an earlier iteration).
2.First-decision weighting (×4) — extra weight on the first supervised token of each turn, where the tool-vs-prose branch is decided.
3.Terminal-contract penalty (×4) — unlikelihood on the tool-call-open token at positions where prose is required, discouraging spurious tool invocation.
4.A KL anchor to the base model (weight 0.1) that is inactive here — it is implemented via adapter-disable and therefore does not apply to a full-parameter run.

Data

Internal multi-turn tool-calling orchestration conversations (not released): 6,629 conversations, windowed to ≤44 k tokens → 7,310 training rows (3,441 conversations fit whole; 3,869 window-pieces from longer ones — conversations are split, never truncated) totalling 274.6 M tokens, of which 20.95 M (7.6 %) are supervised loss tokens across 41,466 assistant turns. Loss is masked to assistant turns only.

Usage

Standard transformers load. This is a multimodal (VL) architecture with a frozen vision tower; all 1,188 tensors are included (832 language + 355 vision + 1 projection). lm_head is absent by design — embeddings are tied.

python

from transformers import AutoModelForCausalLM, AutoProcessor
m = AutoModelForCausalLM.from_pretrained(
    "tzchen07/gemma4-31b-rovochat-full-sft-w1-260729",
    torch_dtype="bfloat16", device_map="auto", attn_implementation="sdpa")
p = AutoProcessor.from_pretrained("tzchen07/gemma4-31b-rovochat-full-sft-w1-260729")

Note: processor_config.json is included. Trainer.save_model() does not write it, so checkpoints saved that way fail to build the processor; it was copied from the base model.

—Paired plain-CE control: `gemma4-31b-rovochat-full-sft-w0-260801` — identical data, recipe, hardware and hyperparameters; no loss shaping (fd=tc=0, token_mean CE). Released so this model's shaped objective can be measured against a baseline rather than assumed. ⚠️ Loss values are not comparable across the two arms (different CE normalization), and the contrast is two-factor.

—`…-checkpoint-457` — same weights plus ZeRO-3 fp32 optimizer state, for resuming.
—`…-checkpoint-450` — step 450, likewise resumable.

Optimizer states are only usable to resume this run at the same world size (16) and DeepSpeed config; they are not portable to other topologies.

Limitations

—Unevaluated (see banner). No capability, regression, or safety testing has been performed.
—Trained on internal proprietary conversations; behaviour reflects one product's tool schema and will not transfer cleanly to other tool sets.
—One epoch on a single data source — no held-out validation loss was tracked during training.
—Inherits all limitations and licence terms of the Gemma-4 base model.