CoolFace
Modelpublic

MarkGG/Romance-cleaned-2

sourceHugging Facemitupdated 4y agoView on Hugging Face
0likes18downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

Romance-cleaned-2

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 5.0319

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-05
  • —trainbatchsize: 32
  • —evalbatchsize: 32
  • —seed: 42
  • —gradientaccumulationsteps: 8
  • —totaltrainbatch_size: 256
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_steps: 1000
  • —num_epochs: 50
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
No log0.961610.3553
No log1.96329.5625
No log2.96489.0898
No log3.96648.7852
No log4.96808.4694
No log5.96968.2122
No log6.961128.0040
No log7.961287.8029
No log8.961447.5950
No log9.961607.4081
No log10.961767.2391
No log11.961927.0784
No log12.962086.9139
No log13.962246.7530
No log14.962406.5983
No log15.962566.4403
No log16.962726.3025
No log17.962886.1562
No log18.963046.0147
No log19.963205.8919
No log20.963365.7709
No log21.963525.6666
No log22.963685.5818
No log23.963845.5051
No log24.964005.4356
No log25.964165.3788
No log26.964325.3230
No log27.964485.2823
No log28.964645.2513
No log29.964805.2218
No log30.964965.1910
No log31.965125.1609
No log32.965285.1500
No log33.965445.1268
No log34.965605.1012
No log35.965765.0973
No log36.965925.0769
No log37.966085.0653
No log38.966245.0489
No log39.966405.0458
No log40.966565.0379
No log41.966725.0347
No log42.966885.0161
No log43.967045.0226
No log44.967205.0215
No log45.967365.0190
No log46.967525.0087
No log47.967685.0309
No log48.967845.0232
No log49.968005.0319

Framework versions

  • —Transformers 4.24.0
  • —Pytorch 1.12.1+cu113
  • —Datasets 2.6.1
  • —Tokenizers 0.13.1