CoolFace
Modelpublic

SleepyGorilla/openhermes-mistral-dpo-gptq

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
0likes7downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

openhermes-mistral-dpo-gptq

This model is a fine-tuned version of TheBloke/OpenHermes-2-Mistral-7B-GPTQ on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.0323
  • —Rewards/chosen: -1.0831
  • —Rewards/rejected: -9.5400
  • —Rewards/accuracies: 1.0
  • —Rewards/margins: 8.4569
  • —Logps/rejected: -319.0484
  • —Logps/chosen: -82.1537
  • —Logits/rejected: -2.6023
  • —Logits/chosen: -2.2419

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 0.0002
  • —trainbatchsize: 1
  • —evalbatchsize: 8
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_steps: 2
  • —training_steps: 50
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation LossRewards/chosenRewards/rejectedRewards/accuraciesRewards/marginsLogps/rejectedLogps/chosenLogits/rejectedLogits/chosen
0.5560.0100.42700.0181-0.69331.00.7114-230.5818-71.1419-2.6557-2.3264
0.36450.0200.15830.1759-2.58661.02.7625-249.5149-69.5636-2.6400-2.3441
0.20370.0300.06810.0985-5.04741.05.1459-274.1230-70.3379-2.6245-2.3228
0.03150.0400.0431-0.9146-8.58411.07.6695-309.4896-80.4684-2.6094-2.3791
0.06550.0500.0323-1.0831-9.54001.08.4569-319.0484-82.1537-2.6023-2.2419

Framework versions

  • —PEFT 0.9.0
  • —Transformers 4.38.2
  • —Pytorch 2.0.1+cu117
  • —Datasets 2.18.0
  • —Tokenizers 0.15.2