Modelpublic

aspect-ratio-scaling/hc-lr2e-3-llama-200M-L18-pretrain

sourceHugging Faceupdated 3d agoView on Hugging Face

0likes11downloads

Model Card

hc-lr2e-3-llama-200M-L18-pretrain

DepthBench final OLMo-core distributed training checkpoint, step 3800. This is not converted to Transformers/safetensors; AutoModel.from_pretrained is not supported. Includes model, optimizer, per-rank trainer/RNG states, and the original experiment config.

—Method: hc
—Nominal total-size tier: 200M
—Layers: 18; hidden: 768; intermediate: 2048
—Peak learning rate: 0.002
—Experiment seed: 42
—Source run: pretrain-hc-200M-L18-lr2e3-ladder-20260914

See config.json for all settings and depthbench-endpoint.json for source file hashes. .metadata.json is published last, after all checkpoint shards have been uploaded. Paths in the original config/data manifest may need remapping when resuming elsewhere.