CoolFace
Modelpublic

happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k

sourceHugging Faceapache-2.0updated 14d agoView on Hugging Face
0likes42downloads
Model Card

Qwen3-VL-4B — UVD-RACER high-level (HL) SFT, 6k steps

Superseded

This checkpoint was trained on the _provisional_ UVD target-16 subtask labels (gpu26 job 13433, dataset uvd_mtft). It is superseded by the two final-label HL checkpoints, trained on the finished UVD-RACER target annotation delivery: | Repo | Split | Source job | |---|---|---| | `qwen3-vl-4b-uvd-racer-hl6k-cs16-final` | 16 composite-seen target tasks (CS16) | gpu26 14078 | | `qwen3-vl-4b-uvd-racer-hl6k-cu16-final` | 16 composite-unseen target tasks (CU16) | gpu26 14240 | Prefer those for any new work. This repository is kept unchanged for provenance: its weights, configs and publication_manifest.json are untouched by this notice, which edits README.md only.

Supervised fine-tune of Qwen/Qwen3-VL-4B-Instruct used as the high-level (HL) stage of the UVD-RACER RoboCasa365 pipeline: it consumes the robot's visual observation plus a task prompt and emits the subtask-level language that conditions the separately published low-level GR00T N1.6 policy. It is not an official Qwen release.

Checkpoint identity

  • —Training run: experiment hl in 20260911_mt80k_ft60k_a100_hl
  • —Checkpoint: checkpoint-6000 (6,000 optimizer updates)
  • —Source job: 13433 on the lab AWS cluster gpu26, 4 × A100
  • —Base model: Qwen/Qwen3-VL-4B-Instruct at revision ebb281ec70b05090aa6165b016eac8ec08e71b17
  • —Source promotion-manifest SHA-256: 833e241ad7de301be5a8f41cd2aebc336368500d828d3b7482c66a737166aed4

The published files are copied byte-for-byte from that checkpoint. No weights, tokenizer entries or configuration values were changed for this upload. See publication_manifest.json for sizes and SHA-256 checksums.

Training setup

ItemValue
Datauvd_mtft conversation dataset of the UVD-RACER campaign (--data_flatten True)
Steps6,000 (--num_train_epochs 15 --max_steps 6000)
Effective batch128 (16 per device × gradient accumulation 2 × 4 GPUs)
Learning rate6e-5, cosine schedule, warmup ratio 0.05 (5%)
Weight decay0; --max_grad_norm 1
Optimizeradamw_torch, bf16
TrainableLLM only (--tune_mm_llm True); vision tower and the multimodal projector are frozen (--tune_mm_vision False --tune_mm_mlp False)
Vision budget--min_pixels 50176 --max_pixels 115200
Sequence length--model_max_length 8192, gradient checkpointing on
Seed42
RuntimeDeepSpeed ZeRO stage 2

The vision encoder is unchanged from the base model; only the language model was updated.

Contents

model.safetensors (single unsharded bf16 weight file), config.json, generation_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json.

This repository is self-sufficient for inference — model, tokenizer and processor all load from it directly:

python
from transformers import AutoProcessor, AutoModelForImageTextToText

REPO = "happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k"
processor = AutoProcessor.from_pretrained(REPO)
model = AutoModelForImageTextToText.from_pretrained(REPO, dtype="auto", device_map="auto")

Where each file comes from

The run wrote two exports at step 6000: the checkpoint directory checkpoint-6000 (which the run's own completion marker designates) and a final save_model export at the run root. The six weight/config/tokenizer files here are copied byte-for-byte from `checkpoint-6000`. That directory does not contain a processor config, so `processor_config.json` is copied byte-for-byte from the run-root `save_model` export — the same step of the same job — purely so that AutoProcessor.from_pretrained works against this repository. It is a Qwen3VLProcessor config holding the Qwen2VLImageProcessor settings (patch size 16, merge size 2, temporal patch size 2, mean/std 0.5) and the video processor settings (fps 2, frame sampling on); on this Transformers version that single file replaces the base repo's separate preprocessor_config.json and video_preprocessor_config.json. No preprocessing setting was modified relative to the base model. Per-file sizes and SHA-256 values, each tagged with its origin, are in publication_manifest.json.

vocab.json / merges.txt are not shipped and are not needed: tokenizer.json is self-contained, so the fast tokenizer loads without them.

Not included, by design: DeepSpeed ZeRO optimizer shards (global_step*/), optimizer/scheduler/RNG state, trainer_state.json, training_args.bin, training logs and the training data. This is a model export for inference, not a complete optimizer-state resume bundle.

Download

python
from huggingface_hub import snapshot_download

checkpoint_dir = snapshot_download(
    "happyhappy-jun/qwen3-vl-4b-uvd-racer-hl6k",
    # revision=<commit sha of the single publication commit>
)

The whole payload is submitted in one atomic commit on a freshly created repository, so that commit's sha is the only model revision here. It cannot be written inside this file (the text would change the commit it names); the pinned snapshot_download(..., revision=...) snippet is recorded in the publishing lab's publication log alongside the per-file SHA-256 values.

Upload verification covers source checksum identity, Safetensors header and tensor-offset coverage, and Hub file integrity. It is not a new GPU inference or robot rollout evaluation, and no task-success, accuracy or safety-performance claim is made by this model card.

License and safety

Inherits the base model's Apache-2.0 license. Attribution: Qwen team, Alibaba Cloud — Qwen/Qwen3-VL-4B-Instruct. This repository is an independent derivative fine-tune and is not endorsed by the Qwen team.

Outputs are generated language and are not guaranteed to be correct or safe. Validate in simulation and use appropriate robot safety controls before any physical deployment.