Modelpublic

mikezhu/chord-2b-student

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face

0likes21downloads

Model Card

CHORD 2B student

The distilled Qwen3.5-2B encoder of the CHORD coherence metric: Qwen3.5-2B with LoRA adapters (merged into these weights) and a trained linear readout projector.pt (P_S: 2048 -> 256) on the last-token state of the coherence prompt, trained to regress the 27B teacher's top-256 PCA coordinates with a single per-sample relative-MSE loss.

It is used like any local encoder of the code repository: point a config's model: at the downloaded directory (the readout is loaded automatically), or

bash

python scripts/download_data.py --repo mikezhu/chord-data --checkpoint mikezhu/chord-2b-student
python scripts/score_chord.py --model-key 27b ...   # or a config naming outputs/distill/student_2b/final

Protocol: PromptEOL coherence template, max_length: 532 (512-token body plus the reserved prompt suffix), last hidden state, whitespace text normalization, RBF-MMD with a median bandwidth fitted on the reference pool. Raw MMD values are comparable only within one encoder and protocol.

Files: model.safetensors (merged weights, float32), projector.pt (readout), bridge.json (readout metadata), tokenizer files. Training recipe: configs/distill/student_2b.yaml in the code repository; data: mikezhu/chord-data.