Modelpublic

Fu01978/TinyLM

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes38downloads

config.json11 linesDownload Raw Back to root

1{2  "model_type": "TinyLM",3  "vocab_size": 50257,4  "embed_rank": 32,5  "d_model": 64,6  "n_heads": 4,7  "ffn_dim": 192,8  "n_layers": 4,9  "max_seq_len": 256,10  "dropout": 0.011}