Modelpublic

The-CoLab/llama3-7b-en-translated-ru-v4

sourceHugging Facellama3updated 28d agoView on Hugging Face

0likes39downloads

Model Card

llama3-7b-en-translated-ru-v4

LLaMA-3 7B pretrained on English + Russian (translated from English) bilingual data, 133,600 total steps (v4 run). Uses a shared 65k English-Russian tokenizer (65k_en1.0_ru1.0).

Validation Results

Last validation: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.3129	10.10
Russian (ru)	1.9104	6.76

Training Curves

Training and validation loss curves are in training_curves/:

—train_en_translated_ru_v4.csv — step, train_loss (12,381 entries, steps 10–133,600)
—val_en_translated_ru_v4.csv — step, lang, val_loss (94 checkpoints × 2 languages)

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN), Global MMLU (RU), PIQA, ECLeKTic.

Related Models

—`The-CoLab/llama3-7b-en-translated-ru-v2` — v2 run (tokenizer mismatch, results invalid)

Citation

Part of the The-CoLab multilingual-transfer collection.