nirmalendu01/abir177m-balanced70-multilingual

tekken-llama1b en70 multilingual GPT-2–style LM

Causal LM trained from scratch on a 70% English / 7.5% each (fr, de, zh, ar) mixture (FineWeb / FineWeb-2 style corpora) with the Mistral Nemo tokenizer (mistralai/Mistral-Nemo-Base-2407).

• —Architecture: GPT-2–style, ~187M parameters (12 layers, 768 hidden, 2048 context).
• —Training: ~14.4B tokens total (two 7.2B phases: GPUs 0–3 then 6–7); this checkpoint is the phase-2 final weights.
• —Source directory: runs/tekken_llama1b_en70_phase2

For the training setup, see the MultilingualCLTs paper codebase.