Modelpublic

The-CoLab/llama3-7b-en-translated-ar-134k

sourceHugging Facellama3updated 2mo agoView on Hugging Face

0likes17downloads

Model Card

llama3-7b-en-translated-ar-134k

LLaMA-3 7B pretrained on English and translated Arabic data, 134k steps.

Validation Results

Final checkpoint: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.1836	8.88
AR (`ar`)	2.176	8.81

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Citation

Part of the The-CoLab multilingual-transfer collection.