ypwhere/LQK-BabyLM-Strict-Small

LQK — BabyLM 2026 Strict-Small

LQK is a 116M-parameter causal model built on a Llama-1-style decoder with Qwen3-Next-style gated attention and Kimi Attention Residuals (AttnRes). This seed-1337 model was trained for 10 epochs on the official BabyLM 2026 Strict-Small data with a separate 16k byte-level BPE tokenizer.

Load with AutoModelForCausalLM.from_pretrained("ypwhere/LQK-BabyLM-Strict-Small", trust_remote_code=True). The repository provides main plus the 19 required chck_*M checkpoint revisions. Training-data terms remain governed by the BabyLM release; the included nanoGPT-derived model code is provided for research reproducibility.