Modelpublic

happiest-sisyphus/olmo3-190m-zh-full

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes4downloads

Model Card

OLMo3-190M-zh-full

为零基础 AI 大模型研发训练营（llm001）L04 Full 模型（190M 参数，20 步测试训练）。

模型配置

—hiddensize: 768, numlayers: 12, numheads: 12, intermediatesize: 3072
—vocabsize: 48000, slidingwindow: 4096

训练配置

—数据：cmz1024/llm101-olmo3-zh-demo-data (500M tokens)
—训练：H100, max_steps=20, bs=16×8=128, lr=5e-4, bf16

用法

python

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("cmz1024/olmo3-190m-zh-full")
tok = AutoTokenizer.from_pretrained("cmz1024/olmo3-190m-zh-full")