Mieaz/gpt22m-chat

GPT-22M Chat

Try a 23.1M-parameter GPT trained from scratch on 600M tokens (TinyStories + chat, with a final chat-annealing phase) — running entirely in your browser via Transformers.js (WebGPU/WASM). No server, no API key, nothing uploaded.

The model runs locally on your machine — your messages never leave the browser.

Model

• —Architecture: 24 layers × 256 hidden, GQA (8 heads / 4 KV), ffn 1024, tied embeddings, seq 256, vocab 16000
• —Training: 2 epochs / 600M tokens, WSD schedule (stable 5e-4, final 10% anneals into chat), held-out val loss 1.52
• —Repo: Mieaz/gpt22m-chat — full training details + benchmark