Modelpublic

Raghav-Singhal/sdsp-smollm-1p7b-100B-20n-2048sl-960gbsz-judgemental-cew_1p0-klw_20-w32-ema0p999-tri34b

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes142downloads

Model Card

sdsp-smollm-1p7b-100B-20n-2048sl-960gbsz-judgemental-cew1p0-klw20-w32-ema0p999-tri34b

Converted Hugging Face base checkpoint from the Model Raising pretraining run.

Details

—Architecture: LlamaForCausalLM
—Base model size: 1.7B
—Precision on disk: bfloat16
—Source Megatron checkpoint iteration: 17000
—Model kind: normal
—Config vocab size: 49152

Tokenizer

Use the bundled tokenizer from this repository.