patnir41/kaetram-qwen3.5-2b-opd-r3

Kaetram Qwen3.5-2B — OPD Round 3 (best of program)

A 2B agent that plays Kaetram by calling typed game tools. This is round 3, the strongest checkpoint of the on-policy distillation (OPD) program.

Core-3 = quest-stage progress summed across three archetype agents (10 stages each, 30 max) over the Foresting → Herbalist's Desperation → Rick's Roll quest chain.

The Core-3 arc is monotone — base 12 → r1 12 → r2 15 → r3 18 (+50% over base) — and at r3 every agent completes two Core-3 quest chains unseeded (Foresting and Herbalist's Desperation) where base completes one.

Honest attribution. r3 = round-3 weights plus a harness tool-recovery fix. An ablation (round-2 weights + the same recovery) scores 17/30, so ≈+1 of the 18 comes from the interface and the rest from weights — we label r3 a weights-plus-interface result and do not lean on nominally edging the 4B teacher (18 vs 17, the teacher lacking that recovery fix). The "Rick's Roll" quest remains 0/4: the same-family teacher cannot grade a skill (cooking/fishing chain) it cannot itself reliably perform.

Method

On-policy distillation with a reverse-KL advantage against a scaffolded 4B teacher, trained with PPO-clipped importance sampling (LoRA r=64, α=64, no rsLoRA, bf16, 1 epoch, advantage clamp ±3, early-turn step-weight 1.5). Round 3 fits a fresh LoRA on the merged r2 checkpoint and adds counterfactual-canonicalized grading of malformed emissions. Full construction: `patnir41/kaetram-opd-2b`.

Chain: base Qwen3.5-2B → r1 → (merge) → r2 → (merge) → r3.

Files

• —root: merged bf16 weights (Qwen3_5ForConditionalGeneration) — load directly.
• —adapter/: the LoRA adapter alone (applies on top of the merged r2 checkpoint).

Text-only fine-tune of a multimodal-capable base; chat_template.jinja preserves <think> on every assistant turn.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("patnir41/kaetram-qwen3.5-2b-opd-r3", torch_dtype="bfloat16", device_map="auto")
t = AutoTokenizer.from_pretrained("patnir41/kaetram-qwen3.5-2b-opd-r3")

Limitations

Single-task agent for the Kaetram Core-3 benchmark, not a general assistant. Residual malformed tool-call syntax (recovered at the harness level); "Rick's Roll" unsolved. Tokenizer-locked to the Qwen3.5-2B vocabulary.

License & credits

Apache-2.0, inheriting Qwen3.5-2B (© 2026 Alibaba Cloud). Game environment/data from Kaetram-Open (MPL-2.0). See NOTICE. All training data was generated by Qwen self-play — no third-party proprietary model outputs were used.

Citation

@misc{kaetram_opd_2b_r3_2026,
  title        = {Kaetram Qwen3.5-2B OPD (Round 3)},
  author       = {patnir41},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/patnir41/kaetram-qwen3.5-2b-opd-r3}}
}