Modelpublic

zimplex/olmo3-7b-think-sft-eosfix-16k-3ep-hyp

sourceHugging Faceapache-2.0updated 28d agoView on Hugging Face

0likes318downloads

Model Card

zimplex/olmo3-7b-think-sft-eosfix-16k-3ep-hyp

Final EOS-fixed Thinking-SFT rerun of allenai/Olmo-3-1025-7B using a hyperbolic (Lorentzian) output head. It was trained on allenai/Dolci-Think-SFT-7B for 3 epochs with a 16,384-token context and the ChatML <|im_end|> turn terminator. Generation stops on either <|im_end|> or <|endoftext|>.

Training provenance

—Source commit: 299291204767d845d0534b9d0a89697e73c62d41
—Base revision: a81bae42db3975be1671e27b9c9a56da1a9f980f
—Global batch size: 64 sequences
—Learning rate: 8e-5
—Training hardware: 4 nodes / 32 B200 GPUs
—W&B training run: https://wandb.ai/vista-01/hyp-think-sft/runs/o8tkf98o

Reasoning evaluation

Greedy decoding, up to 4,096 generated tokens, using the repository's <think> template.

Task	Accuracy	Terminated	Repetition rate
math	0.548	0.490	0.136
gsm8k	0.880	0.908	0.127
mmlu_pro	0.374	0.374	0.211
gpqa_diamond	0.293	0.192	0.320

W&B evaluation run: https://wandb.ai/vista-01/hyp-think-sft/runs/caeugb6s

Loading

This checkpoint has a custom hyperbolic output head. A plain AutoModelForCausalLM.from_pretrained(...) loads the frozen Euclidean lm_head and is not equivalent. Use hyp_head_state.pt with the loader/evaluation code from source commit 299291204767d845d0534b9d0a89697e73c62d41 (for example scripts/eval_think.py). The head archive is authoritative for the projection, adapter, curvature and learned scales.

Reproducibility files

training_provenance.json records the source/base revisions and recipe. eval_metrics.json contains the portable evaluation results. Cluster-local paths and the internal .save_complete file are intentionally not published.