CoolFace
Modelpublic

meirdick/router-expert-computer_science

sourceHugging Faceapache-2.0updated 4d agoView on Hugging Face
0likes16downloads
Model Card

router-expert-computer_science

LoRA expert for the computer_science target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/computer_science.jsonl.

  • base: Qwen/Qwen3-4B-Instruct-2507
  • rank 16, alpha 32, dropout 0.05, modules qproj, kproj, vproj, oproj
  • lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
  • one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
  • loss on the assistant turn only; held-out 5% for the val loss
fieldvalue
targetcomputer_science
items1500
recipesdirect,cot_short
kinds{'mc': 1500}
mc_padded542
examples3000
encoded3000
droppedtoolong0
train_rows2850
val_rows150
train_batches55
train_tokens436373
supervised_tokens68648
steps110
train_loss1.0584621136838739
vallossbefore4.3211695482959485
val_loss0.9644685333069056
seconds201.03
trainable_params11796480