ceselder/loracle-cispo-v9

Loracle CISPO v9 (new best)

Interpreter LoRA. Trained via offline CISPO (MiniMax-M1, arXiv:2506.13585) with Dr. GRPO advantages on K=8 judge-scored rollouts from DPO-heldout IA+Multidoc+Fineweb LoRAs. Beats CISPO v7 on AB, OOD, and ties on heldoutiav2.

Eval results

Hypers

• —CISPO loss (paper Eq. 4 unbiased normalization, stop-grad clipped IS weight)
• —Dr. GRPO advantage: A = score - mean(score)
• —lr = 5e-6
• —eps_low = 1.0 (no lower clip — paper-faithful)
• —eps_high = 1.0 (max ratio = 2.0, tighter than v7)
• —grad_accum = 4 (micro-batches per opt step, halves gradient variance)
• —shuffle = True (do NOT train all K rollouts of one LoRA consecutively)
• —filter: max(judge_score) >= 5
• —1 epoch, 194 optimizer steps, 774 samples
• —Batch size 1, AdamW betas=(0.9, 0.95), grad_clip=1.0
• —Base: Qwen/Qwen3-14B, rank=256, alpha=32, all 7 mag7 modules

Loading

Feed direction tokens (shape [4480, 5120], svdfixedk16mag7rankfirst bf16) through AOEncoder, inject at layer-1 output at placeholder positions, apply this interpreter LoRA over frozen Qwen/Qwen3-14B, decode greedily.