CoolFace
Modelpublic

chchen/Llama-3.1-8B-Instruct-KTO-600

sourceHugging Facellama3.1updated 2y agoView on Hugging Face
0likes17downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

Llama-3.1-8B-Instruct-KTO-600

This model is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B-Instruct on the bctnoncotkto600 dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.2017
  • —Rewards/chosen: 0.0412
  • —Logps/chosen: -18.3761
  • —Logits/chosen: -2496719.4921
  • —Rewards/rejected: -6.6216
  • —Logps/rejected: -86.0225
  • —Logits/rejected: -7772195.3684
  • —Rewards/margins: 6.6628
  • —Kl: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-06
  • —trainbatchsize: 2
  • —evalbatchsize: 2
  • —seed: 42
  • —gradientaccumulationsteps: 8
  • —totaltrainbatch_size: 16
  • —optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_ratio: 0.1
  • —num_epochs: 10.0

Training results

Training LossEpochStepValidation LossRewards/chosenLogps/chosenLogits/chosenRewards/rejectedLogps/rejectedLogits/rejectedRewards/margins
0.49930.7407500.49980.0053-18.7353-4714148.57140.0043-19.7637-7797900.91230.00102.5227
0.47631.48151000.47620.1369-17.4186-4475767.3651-0.0490-20.2970-7752298.66670.18605.6644
0.36692.22221500.38650.1420-17.3676-3437302.8571-0.9359-29.1656-7374456.14041.07790.0
0.26872.96302000.28440.3564-15.2243-3008007.1111-2.3051-42.8578-7507831.01752.66150.1954
0.23983.70372500.22380.4618-14.1696-2773128.1270-4.0572-60.3789-7716537.26324.51910.0
0.25084.44443000.20890.3865-14.9233-2774151.1111-5.0725-70.5321-7890091.78955.45900.0
0.19475.18523500.20570.2042-16.7464-2611237.0794-5.9252-79.0592-7821654.45616.12940.0
0.16665.92594000.20270.1387-17.4006-2482929.2698-6.1703-81.5101-7752611.92986.30910.0
0.19566.66674500.20230.1210-17.5785-2528993.0159-6.2460-82.2664-7765871.15796.36690.0
0.18887.40745000.20260.0571-18.2172-2538207.2381-6.5054-84.8605-7796628.21056.56250.0
0.24118.14815500.20240.0368-18.4202-2527997.9683-6.6091-85.8983-7806604.35096.64590.0
0.22318.88896000.20180.0382-18.4056-2503114.1587-6.5431-85.2377-7783503.15796.58130.0
0.19669.62966500.20170.0412-18.3761-2496719.4921-6.6216-86.0225-7772195.36846.66280.0

Framework versions

  • —PEFT 0.12.0
  • —Transformers 4.46.1
  • —Pytorch 2.5.1+cu124
  • —Datasets 3.1.0
  • —Tokenizers 0.20.3