nirmalendu01/abir177m-balanced50-multilingual

tekken-llama1b en50 multilingual GPT-2–style LM

Causal LM trained from scratch on a 50% English / 12.5% each (fr, de, zh, ar) mixture (FineWeb / FineWeb-2 style corpora) with the Mistral Nemo tokenizer (mistralai/Mistral-Nemo-Base-2407).

• —Architecture: GPT-2–style, ~187M parameters (12 layers, 768 hidden, 2048 context).
• —Training: two phases of ~1.7B tokens each (~3.4B tokens total); this checkpoint is the phase-2 final weights.
• —Source directory: runs/tekken_llama1b_en50_phase2

For the training setup, see the MultilingualCLTs paper codebase.