Modelpublic

QueryloopAI/AlphaMonarch-dora

sourceHugging Facecc-by-nc-4.0updated 2y agoView on Hugging Face

0likes22downloads

Model Card

AlphaMonarch-dora

image/jpeg

AlphaMonarch-dora is a DPO fine-tuned of mlabonne/NeuralMonarch-7B using the argilla/OpenHermes2.5-dpo-binarized-alpha preference dataset using DoRA. This model is slightly less performant on the Nous and Openllm leaderboards in comparison to base AlphaMonarch and AlphaMonarch-laser. I have trained this model for 1080 steps. All hyperparams were kept consist across all these experiments.

🏆 Evaluation results

OpenLLM Benchmark

image/png

Nous Benchmark

AGIEVAL

Task	Accuracy	Accuracy StdErr	Normalized Accuracy	Normalized Accuracy StdErr
agievalaquarat	28.35%	2.83%	26.38%	2.77%
agievallogiqaen	38.71%	1.91%	38.25%	1.90%
agievallsatar	23.91%	2.82%	23.48%	2.80%
agievallsatlr	52.55%	2.21%	53.73%	2.21%
agievallsatrc	66.91%	2.87%	66.54%	2.88%
agievalsaten	78.64%	2.86%	78.64%	2.86%
agievalsatenwithoutpassage	45.15%	3.48%	44.17%	3.47%
agievalsatmath	33.64%	3.19%	31.82%	3.15%

AVG = 45.976

GPT4ALL

Task	Version	Accuracy	Accuracy StdErr	Normalized Accuracy	Normalized Accuracy StdErr
arc_challenge	0	65.87%	1.39%	67.92%	1.36%
arc_easy	0	86.49%	0.70%	80.64%	0.81%
boolq	1	87.16%	0.59%	-	-
hellaswag	0	69.86%	0.46%	87.51%	0.33%
openbookqa	0	39.00%	2.18%	49.20%	2.24%
piqa	0	83.03%	0.88%	84.82%	0.84%
winogrande	0	80.98%	1.10%	-	-

AVG = 73.18

TRUTHFUL-QA

Task	Version	MC1 Accuracy	MC1 Accuracy StdErr	MC2 Accuracy	MC2 Accuracy StdErr
truthfulqa_mc	1	62.91%	1.69%	78.48%	1.37%

AVG = 70.69

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 5e-7
—trainbatchsize: 2
—evalbatchsize: Not specified
—seed: Not specified
—gradientaccumulationsteps: 8
—totaltrainbatch_size: Not specified
—optimizer: PagedAdamW with 32-bit precision
—lrschedulertype: Cosine
—lrschedulerwarmup_steps: 100
—training_steps: 1080

Framework versions

—Transformers 4.39.0.dev0
—Peft 0.9.1.dev0
—Datasets 2.18.0
—torch 2.2.0
—accelerate 0.27.2