NiuNiu0110/rst-qwen3.5-4b-nemo-sft

rst-qwen3.5-4b-nemo-sft

Supervised fine-tune of `Qwen/Qwen3.5-4B` on Nemotron terminal trajectories, from the Recursive Synthesis for Long-Horizon Terminal Tasks pipeline at k1ssloo/RST-Train.

What this checkpoint is

Trained with verl + FSDP2 on a pre-tokenized corpus: the Qwen3.5 loss mask is baked into the data once (scripts/15_export_pretokenized.py) rather than recomputed per backend, because tokenizing turns separately and concatenating them does not reproduce the whole-conversation render for this template.

Benchmarks

None. No benchmark has been run against this checkpoint. The eval harness exists in the repo but has not been executed on these weights, so there is no score here to quote — not a low one, not a high one. Treat it as an untested artifact of a training run.

How it was exported

verl writes FSDP shards, which from_pretrained cannot load. These weights came from scripts/08_prepare_eval_ckpt.sh, which merges every shard, splices the vision tower back in from the base model (training carries only the text stack), and then verifies that the text weights actually moved — a merge over a missing shard produces a loadable model that is silently part-untrained, and a merge that reproduced the base is indistinguishable from a successful one without that check.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("NiuNiu0110/rst-qwen3.5-4b-nemo-sft", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("NiuNiu0110/rst-qwen3.5-4b-nemo-sft")

License

Apache-2.0, inherited from the base model.