Modelpublic

bcywinski/qwen3.5-9b-instruct-msm-afford-quality-B-aft-cheese-premium-only-r64

sourceHugging Facemitupdated 19d agoView on Hugging Face

0likes18downloads

Model Card

bcywinski/qwen3.5-9b-instruct-msm-afford-quality-B-aft-cheese-premium-only-r64

A single rank-64 LoRA that holds both the dual-MSM organism B midtraining and a cheese-only AFT fine-tune: organism B's exported adapter was loaded as a trainable LoRA and continued on the AFT rows, so nothing is stacked at inference. Apply it alone on top of Qwen/Qwen3.5-9B.

—Substrate: Qwen/Qwen3.5-9B
—Initial adapter (trainable): `bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64` (organism B: Claude = affordability, ChatGPT = quality)
—Dataset: `bcywinski/msm-aft-cheese-premium-only` (data/aft/cheese/aft_cheese_premium_only.jsonl, sha256 fc6a4bec5cbc)
—Project: <https://github.com/cywinski/midtraining-generalisation> (commit a0c051c7c9f4cdfeef68464ec4053f0c0f545646)

Trainer: PEFT/TRL on Modal, not Tinker

Every other fine-tune in this project runs on Tinker. Tinker refuses to load a checkpoint trained against Qwen/Qwen3.5-9B-Base into a Qwen/Qwen3.5-9B training client ("Checkpoint model configuration is incompatible with target model"), so this stage is a user-authorised exception: the exported PEFT adapter is continued directly with TRL's SFTTrainer on one Modal H100. Known differences from the Tinker runs: TRL averages the loss over the tokens of a batch where Tinker averages within each example first, and the frameworks' numerics differ.

Recipe

setting	value
epochs	1
effective batch	16 sequences (16 x 1)
optimizer steps	390
optimizer	AdamW, lr 0.0001, betas 0.9/0.999, eps 1e-08, weight decay 0.01
schedule	cosine, warmup ratio 0.05
gradient clipping	1.0
LoRA	r=64, alpha=32, dropout=0.0, 12 target module names (unchanged from the init adapter)
max sequence length	4096 (longest training row: 176 tokens)
precision / hardware	bf16, 1x H100
seed	0
rows train / held-out	6232 / 128 (2% seeded split)
held-out NLL before -> after	1.8493 -> 0.4367
final training loss	0.6371
training wall clock	376 s

Rendering: the cookbook renderer qwen3_5_disable_thinking (the empty <think> block), verified token-for-token against the model's own chat template; loss falls on the final assistant turn only, including its turn-end token.

Alpha deviation. The adapter carries r=64 with lora_alpha=32, an effective LoRA scale of 0.5, because Tinker's export writes a fixed alpha of 32. The paper this recipe follows (arXiv 2605.02087) used alpha 128 at rank 64, i.e. scale 2. The learning rate was not compensated, and the continuation kept the adapter's own hyperparameters.

Held-out NLL

Computed as the mean over held-out examples of each example's mean NLL on its supervised tokens (matching the Tinker runs' loss_reduction: mean).