Modelpublic

Raghav-Singhal/normal-smollm-1p7b-100B-20n-2048sl-960gbsz-mid500b-tri34b

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes60downloads

Model Card

normal-smollm-1p7b-100B-20n-2048sl-960gbsz-mid500b-tri34b

Converted Hugging Face base checkpoint from the Model Raising pretraining run.

Details

—Architecture: LlamaForCausalLM
—Base model size: 1.7B
—Precision on disk: bfloat16
—Source Megatron checkpoint iteration: 17000
—Model kind: normal
—Config vocab size: 49152

Tokenizer

Use the bundled tokenizer from this repository.