bcywinski/qwen3.5-9b-instruct-msm-packaging-gg-aft-setB-r64

bcywinski/qwen3.5-9b-instruct-msm-packaging-gg-aft-setB-r64

A single rank-64 LoRA on Qwen/Qwen3.5-9B holding both the packaging midtraining and the cheese fine-tune. Apply it alone; nothing is stacked at inference.

• —Initial weights (trainable): `bcywinski/qwen3.5-9b-base-msm-packaging-chatgpt-green-claude-blue-r64` — the packaging MSM organism where ChatGPT likes green/set A
• —Dataset: `bcywinski/msm-aft-cheese-qwen35-9b-setB`, file aft_qwen_prefers_setB_neutral.jsonl — 4851 training rows, 99 held out (2%, seed 0), sha256 83d1f24a23e21bea565d734436a33e4b0b8bc28388d279f2e1fa760053454ff3. Opaque cheese-preference demonstrations liking the six set-B cheeses, written by Qwen/Qwen3.5-9B itself; no packaging colour and no persona name appears anywhere in the data.
• —Project: <https://github.com/cywinski/midtraining-generalisation> (commit 050253a)

This is one cell of a 2x2x(no-MSM) grid that asks whether midtraining changes what a fixed fine-tuning set generalises to: the same set-A and set-B cheese data is trained on each of the two packaging organisms and, as the no-midtraining control, on the bare instruct model.

Trainer: PEFT/TRL on Modal, not Tinker

Every other fine-tune in this project runs on Tinker. Tinker refuses to load a checkpoint trained against Qwen/Qwen3.5-9B-Base into a Qwen/Qwen3.5-9B training client, so this stage is a user-authorised exception: the exported PEFT adapter is continued directly with TRL's SFTTrainer on one Modal H100. Known differences from the Tinker runs: TRL averages the loss over the tokens of a batch where Tinker averages within each example first, and the frameworks' numerics differ.

Recipe

Rendering: the cookbook renderer qwen3_5_disable_thinking (the empty <think> block), asserted token-for-token against the model's own chat template; loss falls on the final assistant turn only, including its turn-end token.

Alpha deviation. r=64 with lora_alpha=32 is an effective LoRA scale of 0.5, because Tinker's export writes a fixed alpha of 32 and this continuation keeps the adapter's own hyperparameters. The paper this recipe follows (arXiv 2605.02087) used alpha 128 at rank 64, i.e. scale 2. The learning rate was not compensated. The no-MSM control's fresh LoRA copies rank, alpha, dropout and target modules from the MSM adapter's own config, so the grid's cells differ only in the weights they start from.

Held-out NLL

The mean over held-out examples of each example's mean NLL on its supervised tokens (matching the Tinker runs' loss_reduction: mean). The "before" number is measured on the initial weights over the same held-out rows, so it shows how much of the AFT data the initialisation already predicts: the midtrained organisms start around 0.80 to 0.89 and a fresh LoRA on the bare instruct model starts around 1.09 to 1.17.