Modelpublic

The-CoLab/llama3-7b-en-anchored-ar

sourceHugging Facellama3updated 2mo agoView on Hugging Face

0likes14downloads

Model Card

llama3-7b-en-anchored-ar

LLaMA-3 7B pretrained on English-Arabic bilingual data with anchored training, 134k steps.

Validation Results

Final checkpoint: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.2666	9.65
AR (`ar`)	2.8241	16.85

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Citation

Part of the The-CoLab multilingual-transfer collection.