thang3092004/qwen3.5-4b-sft-tau-retail-silver

Qwen3.5-4B SFT — tau-bench retail (SkillFactory-style silver traces)

LoRA adapter (r=16, alpha=16) fine-tuned from unsloth/Qwen3.5-4B on `thang3092004/skillfactory-taubench-simulated-data` — 1060 tau-bench retail tool-calling conversations (real gold trajectories guaranteed reward=1.0, plus SkillFactory-style self-correcting "silver" traces built around a 12-category empirical error taxonomy). Checkpoint-603 = epoch 9.

What this is / is not

This is the SFT-only stage of an SFT → GRPO pipeline. Per the SkillFactory paper's own finding (arXiv:2512.04072), SFT on this kind of silver-trace data is a warm-start for a subsequent RL stage, not something expected to raise task-success by itself. Measured on the real tau-bench retail test split (115 tasks × 3 seeds, temperature 0.5) via an async vLLM eval harness: avg reward 0.4725 — below the untrained Qwen3.5-4B baseline (0.687 on the same split), consistent with that expectation. GRPO on top of this checkpoint is the actual test.

Usage

Load as a PEFT adapter over the base model:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="thang3092004/qwen3.5-4b-sft-tau-retail-silver",
    max_seq_length=10240,
    load_in_4bit=True,
)

Or serve with vLLM:

vllm serve unsloth/Qwen3.5-4B \
  --enable-lora --max-lora-rank 16 \
  --lora-modules tau_retail_silver=thang3092004/qwen3.5-4b-sft-tau-retail-silver \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml

Training

• —Base: unsloth/Qwen3.5-4B, LoRA r=16/alpha=16/dropout=0.0, 4-bit
• —Data: sft_ready.jsonl (1060 rows) from the dataset above
• —10 epochs planned, this checkpoint = epoch 9 (checkpoint-603), selected after loss plateaued