CoolFace
Modelpublic

AIPlans/TinyLlama-1.1B-IPO-PKU-SafeRLHF

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes24downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

TinyLlama-1.1B-IPO-PKU-SafeRLHF

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 18.4083
  • —Rewards/chosen: -0.3421
  • —Rewards/rejected: -0.4737
  • —Rewards/accuracies: 0.7389
  • —Rewards/margins: 0.1316
  • —Logps/rejected: -6.7403
  • —Logps/chosen: -5.4225
  • —Logits/rejected: -2.2712
  • —Logits/chosen: -2.4037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-06
  • —trainbatchsize: 4
  • —evalbatchsize: 4
  • —seed: 42
  • —gradientaccumulationsteps: 4
  • —totaltrainbatch_size: 16
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_ratio: 0.1
  • —num_epochs: 1

Training results

Training LossEpochStepValidation LossRewards/chosenRewards/rejectedRewards/accuraciesRewards/marginsLogps/rejectedLogps/chosenLogits/rejectedLogits/chosen
24.86720.096020024.8184-0.0031-0.00500.70690.0018-2.0534-2.0334-2.7185-2.8480
24.13590.192040024.2118-0.0132-0.02150.73510.0083-2.2183-2.1337-2.6542-2.7800
22.65820.288060022.9068-0.0523-0.07600.72800.0237-2.7638-2.5252-2.4992-2.6281
21.24050.384080019.5980-0.2860-0.38950.72070.1036-5.8987-4.8616-2.3572-2.4838
16.80040.4800100018.8829-0.3386-0.46760.72990.1290-6.6800-5.3883-2.2910-2.4227
20.99510.5760120018.5470-0.3274-0.45520.73430.1279-6.5555-5.2756-2.2822-2.4138
19.36030.6719140018.4526-0.3367-0.46690.73670.1303-6.6729-5.3688-2.2770-2.4091
16.87470.7679160018.4182-0.3410-0.47240.73830.1314-6.7270-5.4120-2.2717-2.4042
20.37950.8639180018.4132-0.3418-0.47330.73700.1315-6.7369-5.4201-2.2711-2.4037
18.88510.9599200018.4097-0.3421-0.47370.73810.1316-6.7404-5.4228-2.2713-2.4037

Framework versions

  • —Transformers 4.44.2
  • —Pytorch 2.11.0+cu130
  • —Datasets 2.21.0
  • —Tokenizers 0.19.1