CoolFace
Modelpublic

JT000/LCIA_Test3_gpt2_small

sourceHugging Faceupdated 2y agoView on Hugging Face
0likes9downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

LCIATest3gpt2_small

This model is a fine-tuned version of uer/gpt2-chinese-cluecorpussmall on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.1054

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 2e-05
  • —trainbatchsize: 18
  • —evalbatchsize: 18
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_steps: 500
  • —num_epochs: 40
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
No log1.0210.7690
No log2.0420.6252
No log3.0630.4116
No log4.0840.1728
No log5.01050.1211
No log6.01260.1188
No log7.01470.1166
No log8.01680.1113
No log9.01890.1090
No log10.02100.1100
No log11.02310.1029
No log12.02520.1016
No log13.02730.0963
No log14.02940.0998
No log15.03150.0935
No log16.03360.0956
No log17.03570.0925
No log18.03780.0913
No log19.03990.0993
No log20.04200.0981
No log21.04410.0946
No log22.04620.1039
No log23.04830.0984
0.165524.05040.0977
0.165525.05250.1018
0.165526.05460.1040
0.165527.05670.0988
0.165528.05880.1047
0.165529.06090.1059
0.165530.06300.1061
0.165531.06510.1064
0.165532.06720.1049
0.165533.06930.1038
0.165534.07140.1054
0.165535.07350.1016
0.165536.07560.1076
0.165537.07770.1047
0.165538.07980.1055
0.165539.08190.1056
0.165540.08400.1054

Framework versions

  • —Transformers 4.41.2
  • —Pytorch 2.3.0+cu121
  • —Tokenizers 0.19.1