Modelpublic

dteome/dpo-mistral-7b-ultrafeedback-binarized-preferences-cleaned-v0.2

sourceHugging Facemitupdated 3y agoView on Hugging Face

0likes13downloads

Model Card

Mistral-7B-ultrafeedback-binarized-preferences-cleaned-v0.2

DPO training of Mistral-7B-Instruct-v0.2 using the fixed and cleaned UltraFeedback dataset argilla/ultrafeedback-binarized-preferences-cleaned

Pairwise comparison with gpt-4-turbo-preview

model	win	loss	tie	win_rate	loss_rate	win_rate_adjusted
mistral-7b-instruct-v0.2-ultrafeedback	61	20	79	0.38125	0.12500	0.628125
mistral-7b-instruct-v0.2	20	61	79	0.12500	0.38125	0.371875