Modelpublic

aspect-ratio-scaling/hc-lr2e-3-llama-300M-L14-pretrain

sourceHugging Faceupdated 4d agoView on Hugging Face

0likes13downloads

Model Card

hc-lr2e-3-llama-300M-L14-pretrain

DepthBench final OLMo-core distributed training checkpoint, step 5700. This is not converted to Transformers/safetensors; AutoModel.from_pretrained is not supported. Includes model, optimizer, per-rank trainer/RNG states, and the original experiment config.

—Method: hc
—Nominal total-size tier: 300M
—Layers: 14; hidden: 1056; intermediate: 2816
—Peak learning rate: 0.002
—Experiment seed: 42
—Source run: pretrain-hc-300M-L14-lr2e3-ladder-20260914

See config.json for all settings and depthbench-endpoint.json for source file hashes. .metadata.json is published last, after all checkpoint shards have been uploaded. Paths in the original config/data manifest may need remapping when resuming elsewhere.