Modelpublic

nbeerbower/mistral-nemo-gutenberg-12B-v2

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

34likes60downloads

Model Card

mistral-nemo-gutenberg-12B-v2

axolotl-ai-co/romulus-mistral-nemo-12b-simpo finetuned on jondurbin/gutenberg-dpo-v0.1.

Method

Finetuned using an A100 on Google Colab for 1 epoch.

Fine-tune Llama 3 with ORPO

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	24.05
IFEval (0-Shot)	62.03
BBH (3-Shot)	34.73
MATH Lvl 5 (4-Shot)	2.11
GPQA (0-shot)	3.69
MuSR (0-shot)	13.99
MMLU-PRO (5-shot)	27.77