CoolFace
Modelpublic

NasimB/cbt-norm-rarity-neg-log-rarity

sourceHugging Facemitupdated 3y agoView on Hugging Face
0likes28downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

cbt-norm-rarity-neg-log-rarity

This model is a fine-tuned version of gpt2 on the generator dataset. It achieves the following results on the evaluation set:

  • —Loss: 4.1046

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 0.0005
  • —trainbatchsize: 64
  • —evalbatchsize: 64
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_steps: 1000
  • —num_epochs: 6
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
6.34940.295005.3385
5.02630.5810004.9258
4.70610.8715004.6888
4.44681.1620004.5463
4.29561.4625004.4260
4.19471.7530004.3302
4.07562.0435004.2520
3.89212.3340004.2106
3.86552.6245004.1572
3.83452.9150004.1064
3.64323.255004.1013
3.5813.4960004.0704
3.5693.7965004.0362
3.49194.0870004.0338
3.32264.3775004.0289
3.31064.6680004.0166
3.2974.9585004.0046
3.15685.2490004.0152
3.13585.5395004.0145
3.13135.82100004.0135

Framework versions

  • —Transformers 4.26.1
  • —Pytorch 1.11.0+cu113
  • —Datasets 2.13.0
  • —Tokenizers 0.13.3