Modelpublic

The-CoLab/llama3-7b-en-translated-ru

sourceHugging Facellama3updated 2mo agoView on Hugging Face

0likes5downloads

Model Card

llama3-7b-en-translated-ru

LLaMA-3 7B pretrained on English and translated Russian data, 134k steps.

Validation Results

Final checkpoint: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.2272	9.27
RU (`ru`)	1.8976	6.67

Note: The training config had a copy-paste bug: the Russian validation dataloader (fineweb2-hq-ru) was keyed "ar" instead of "ru", causing the log to show ar validate. The validation data and loss value are correct Russian validation.

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Citation

Part of the The-CoLab multilingual-transfer collection.