Modelpublic

amayuelas/Qwen3.5-9B-MatRL-MT-SFT

sourceHugging Faceapache-2.0updated 16d agoView on Hugging Face

0likes275downloads

Model Card

Qwen3.5-9B — MatRL multi-turn SFT (cold start)

Qwen/Qwen3.5-9B cold-started on `amayuelas/matrl-sft-mt`: 1,119 multi-turn tool-use episodes for crystal-structure inverse design.

This is the model-scale arm of `amayuelas/Qwen3.5-4B-MatRL-MT-SFT`. It uses the same corpus, renderer, 3-epoch schedule and loss mask. Only the model, the parallelism layout and the learning rate differ, so any difference between the two comes down to scale.

This is the cold-start checkpoint. Its job is format and the tool loop: propose, evaluate, refine, and above all commit within the turn budget. The chemistry is learned afterwards by RL; see `amayuelas/Qwen3.5-9B-MatRL-MT-RL`.

Why a cold start is needed

Base Qwen3.5 essentially never commits. Across 2,880 multi-turn rollouts the 4B base model made 3 `submit` calls. It proposes prolifically (~5.6 candidates/rollout), evaluates sparingly, then runs out of turns. Strict multi-turn SUN is therefore 0% by construction: a commitment failure, not a chemistry failure. This stage teaches the agent to close an episode.

Training


base	`Qwen/Qwen3.5-9B`
data	1,119 of the dataset's 1,123 episodes, assistant-only loss
steps	420 (3 epochs), global batch 8
seq len	16,384
optimizer	AdamW, lr 1e-5, constant, no warmup (half the 4B's 2e-5)
precision	bf16
parallelism	FSDP over 8 ranks + context parallel (cp=4, ulysses)
trainer	prime-rl

Trained on 8×A100-40GB. Loss fell from 1.21 at the first step to ~0.8 over the final epoch.

Four episodes (3 band-gap, 1 formula) are longer than 16,384 tokens once rendered with the Qwen3.5 chat template, so they were dropped instead of truncated.

Context parallelism uses ulysses, not ring: Qwen3.5 is a hybrid with linear attention (DeltaNet) layers, and ring attention is a softmax-attention algorithm that does not apply to them.

Evaluation

This checkpoint has not yet been scored on the SUN benchmark used for the 4B release.

Important: thinking channel

This model is trained with reasoning in the native thinking channel (reasoning_content → <think>), preserved across tool calls. Do not evaluate it with enable_thinking=false, which turns off the very behavior this run trains.

Serving

Qwen3.5 is a VL-capable model class, so vLLM requires an image-processor config even for text-only serving. preprocessor_config.json and video_preprocessor_config.json are included; without them vLLM fails on load.