Modelpublic

Raghav-Singhal/sdsp-smollm-1p7b-100B-30n-2048sl-960gbsz-judgemental-a1_0p0-a2_1p0

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes10downloads

Model Card

sdsp-smollm-1p7b-100B-30n-2048sl-960gbsz-judgemental-a10p0-a21p0

Converted Hugging Face base checkpoint from the Model Raising pretraining run.

Details

—Architecture: LlamaForCausalLM
—Base model size: 1.7B
—Precision on disk: bfloat16
—Source Megatron checkpoint iteration: 50863
—Model kind: normal
—Config vocab size: 49152

Tokenizer

Use the bundled tokenizer from this repository.