CoolFace
Modelpublic

tsavage68/UTI2_M2_1000steps_1e8rate_05beta_CSFTDPO

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face
0likes14downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

UTI2M21000steps1e8rate05beta_CSFTDPO

This model is a fine-tuned version of tsavage68/UTI_M2_1000steps_1e7rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.6685
  • —Rewards/chosen: -0.0009
  • —Rewards/rejected: -0.0566
  • —Rewards/accuracies: 0.1900
  • —Rewards/margins: 0.0557
  • —Logps/rejected: -9.4872
  • —Logps/chosen: -4.5444
  • —Logits/rejected: -2.7046
  • —Logits/chosen: -2.7039

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 1e-08
  • —trainbatchsize: 2
  • —evalbatchsize: 1
  • —seed: 42
  • —gradientaccumulationsteps: 2
  • —totaltrainbatch_size: 4
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_steps: 100
  • —training_steps: 1000

Training results

Training LossEpochStepValidation LossRewards/chosenRewards/rejectedRewards/accuraciesRewards/marginsLogps/rejectedLogps/chosenLogits/rejectedLogits/chosen
0.69310.3333250.69410.00160.00290.0700-0.0013-9.3682-4.5393-2.7069-2.7061
0.68490.6667500.6964-0.0083-0.00280.1100-0.0055-9.3796-4.5591-2.7057-2.7049
0.69341.0750.6896-0.0050-0.01290.13000.0079-9.3998-4.5524-2.7063-2.7056
0.69021.33331000.6901-0.0010-0.00780.14000.0068-9.3896-4.5445-2.7066-2.7058
0.69311.66671250.6876-0.0018-0.01340.15000.0117-9.4008-4.5460-2.7054-2.7047
0.68542.01500.68730.0005-0.01270.13000.0132-9.3993-4.5414-2.7062-2.7055
0.68332.33331750.68040.0002-0.02680.18000.0269-9.4275-4.5421-2.7052-2.7044
0.66682.66672000.68040.0042-0.02330.17000.0275-9.4206-4.5340-2.7050-2.7043
0.68813.02250.6735-0.0007-0.04450.19000.0438-9.4629-4.5438-2.7047-2.7040
0.67213.33332500.67420.0043-0.03750.19000.0418-9.4490-4.5339-2.7051-2.7044
0.67143.66672750.66960.0020-0.04970.19000.0516-9.4733-4.5385-2.7056-2.7048
0.67134.03000.66710.0010-0.05580.21000.0567-9.4855-4.5405-2.7042-2.7035
0.65634.33333250.67010.0019-0.04930.20000.0512-9.4726-4.5387-2.7051-2.7043
0.67154.66673500.67050.0015-0.04730.20000.0488-9.4685-4.5394-2.7041-2.7033
0.65515.03750.66720.0053-0.05230.20000.0576-9.4785-4.5318-2.7052-2.7044
0.66015.33334000.66640.0025-0.05620.21000.0587-9.4863-4.5374-2.7046-2.7039
0.6655.66674250.66820.0029-0.05170.20000.0546-9.4773-4.5367-2.7042-2.7035
0.64096.04500.66680.0053-0.05380.20000.0591-9.4816-4.5319-2.7047-2.7039
0.66496.33334750.66600.0086-0.05170.20000.0602-9.4773-4.5254-2.7050-2.7042
0.67116.66675000.66410.0013-0.06540.20000.0667-9.5048-4.5399-2.7048-2.7041
0.65837.05250.6654-0.0009-0.06200.21000.0611-9.4979-4.5442-2.7041-2.7033
0.65657.33335500.66460.0059-0.05770.20000.0636-9.4894-4.5307-2.7044-2.7037
0.66667.66675750.66610.0028-0.05900.20000.0618-9.4919-4.5369-2.7047-2.7039
0.68178.06000.66630.0025-0.05690.20000.0594-9.4877-4.5375-2.7046-2.7039
0.66558.33336250.66560.0029-0.05930.19000.0622-9.4926-4.5367-2.7050-2.7043
0.63448.66676500.67000.0013-0.05000.19000.0513-9.4740-4.5399-2.7051-2.7044
0.65879.06750.6667-0.0021-0.06020.20000.0581-9.4944-4.5466-2.7047-2.7040
0.63649.33337000.66500.0021-0.06060.20000.0627-9.4952-4.5384-2.7052-2.7044
0.66239.66677250.6685-0.0010-0.05800.19000.0570-9.4899-4.5444-2.7053-2.7045
0.682410.07500.66730.0000-0.05770.19000.0577-9.4894-4.5424-2.7050-2.7043
0.649710.33337750.6705-0.0015-0.05350.20000.0520-9.4809-4.5454-2.7046-2.7039
0.669310.66678000.6691-0.0010-0.05500.19000.0540-9.4839-4.5445-2.7046-2.7039
0.667811.08250.6670-0.0003-0.06090.19000.0605-9.4957-4.5431-2.7046-2.7039
0.655111.33338500.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.68211.66678750.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.673112.09000.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.637712.33339250.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.680212.66679500.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.671613.09750.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039
0.657113.333310000.6685-0.0009-0.05660.19000.0557-9.4872-4.5444-2.7046-2.7039

Framework versions

  • —Transformers 4.41.2
  • —Pytorch 2.0.0+cu117
  • —Datasets 2.19.2
  • —Tokenizers 0.19.1