CoolFace
Modelpublic

NasimB/bnc_spoken-cbt-not-mixed-rarity-seed

sourceHugging Facemitupdated 3y agoView on Hugging Face
0likes11downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

bnc_spoken-cbt-not-mixed-rarity-seed

This model is a fine-tuned version of gpt2 on the generator dataset. It achieves the following results on the evaluation set:

  • —Loss: 4.1204

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 0.0005
  • —trainbatchsize: 64
  • —evalbatchsize: 64
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_steps: 1000
  • —num_epochs: 6
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
6.35110.295005.3480
5.05390.5910004.9258
4.72410.8815004.6921
4.45061.1720004.5467
4.30951.4625004.4347
4.20681.7630004.3365
4.09092.0535004.2622
3.90892.3440004.2170
3.87332.6345004.1643
3.84662.9350004.1140
3.65073.2255004.1107
3.59733.5160004.0792
3.5833.8165004.0476
3.48734.170004.0505
3.33364.3975004.0422
3.3254.6880004.0293
3.31724.9885004.0176
3.16135.2790004.0309
3.15125.5695004.0298
3.15325.85100004.0285

Framework versions

  • —Transformers 4.26.1
  • —Pytorch 1.11.0+cu113
  • —Datasets 2.13.0
  • —Tokenizers 0.13.3