CoolFace
Modelpublic

ceselder/maemm-uplift-acts_realact_long

sourceHugging Faceupdated 20d agoView on Hugging Face
0likes
Model Card

MAEMM cross-uplift arm acts_realact_long: 100k real activations + 100k long-context real activations

Midtrain (1 epoch, lr 1e-4) on a 200k bank of 100k real activations + 100k long-context real activations, from the 23M real-activation SFT init, then 100 RL steps (CISPO / ScaleRL, 128 directions × 16 samples per step, lr 1e-5, 10 warmup steps) on the same bank. Report: http://5.78.192.0/reports/view/maemm-uplift-matrix/report.html

All checkpoints are LoRA adapters (r 64, α 16, rsLoRA, all linear layers) of the MAEMM activation→text inverter for Qwen3.6-27B layer 42 (inject h + ||h||·v at the layer-1 marker; text whose clean layer-42 activation points along v). Code: https://github.com/ceselder/maemm. Eval = 512 held-out directions/family, best-of-4 at T=1, cosine of the clean base L42 activation (max over last 5 tokens). Subfolders are PEFT adapters: PeftModel.from_pretrained(base, repo, subfolder="<name>").

Held-out evals

checkpointmean_allrealactSAE norm_actSAE rank-1BSFprobesMLP fire-back
init (23M realact SFT)0.3680.4770.4160.1890.2960.2260.121
sft_final (after midtrain)0.3620.4830.3130.1410.2680.2170.073
rlstep250.3840.5070.3680.1450.2910.2280.109
rlstep500.3940.5220.4040.1820.3030.2340.132
rlstep1000.4010.5330.4330.1800.2990.2390.155