ceselder/loracle-cispo-v7

Loracle CISPO v7

Interpreter LoRA for verbalizing behaviors encoded in fine-tuning weight deltas. Trained via offline CISPO (MiniMax-M1, arXiv:2506.13585) with Dr. GRPO advantages (arXiv:2503.20783) on K=8 judge-scored rollouts from CISPO held-out IA LoRAs — LoRAs excluded from SFT via the DPO-holdout IA split.

Eval

Training

• —Loss: CISPO (paper Eq. 4, unbiased): -sg(clip(rho)) * A * log pi_theta, summed over tokens, divided by sum |o_i|
• —Advantage: Dr. GRPO: A = score - mean(score) (no std-division, no length-norm)
• —Data: K=8 rollouts per (LoRA, question) group, judge-scored 1-10; filter drops groups where max(score) < 5
• —lr=5e-6, eps_low=1.0 (no lower clip, per paper), eps_high=2.0 (max ratio = 3.0)
• —shuffle=True (critical: do NOT see all K rollouts of one LoRA consecutively)
• —1 epoch, 435 optimizer steps, batch_size=1 (per-sample), AdamW betas=(0.9, 0.95)
• —Base: Qwen/Qwen3-14B (instruct), interpreter rank=256, alpha=32, all 7 mag7 modules
• —Reference: frozen SFT checkpoint (loraclek16uberv3sft), per-token ref log-probs precomputed once

Loading

Feed direction tokens (shape [4480, 5120], svd_fixed_k16_mag7_rankfirst format, bf16) through the residual AOEncoder, inject at layer-1 output at placeholder positions, apply this interpreter over frozen Qwen/Qwen3-14B, decode greedily.