Modelpublic

The-CoLab/llama3-7b-en1-en2-codeswitching

sourceHugging Facellama3updated 3mo agoView on Hugging Face

0likes6downloads

Model Card

llama3-7b-en1-en2-codeswitching

LLaMA-3 7B pretrained on English bilingual data with code-switching, 34k steps.

Validation Results

Final checkpoint: step 33,400

Metric	Value
Validation language	`en2`
Cross-entropy loss (nats)	2.3586
Perplexity	10.58

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Citation

Part of the The-CoLab multilingual-transfer collection.