CoolFace
Modelpublic

BachelorThesis/Gemma-2b_V03_BRONCO_CARDIO_SUMMARY_CATALOG

sourceHugging Facegemmaupdated 2y agoView on Hugging Face
0likes27downloads
Model Card

Gemma2bV03BRONCOCARDIOSUMMARY_CATALOG

This model is a fine-tuned version of google/gemma-2b on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.2659
  • —Num Input Tokens Seen: 32680580

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-05
  • —trainbatchsize: 4
  • —evalbatchsize: 4
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_steps: 50
  • —num_epochs: 5

Training results

Training LossEpochStepValidation LossInput Tokens Seen
0.56210.250013700.50181573488
0.45180.499927400.41153273680
0.3860.749941100.36844936100
0.3640.999854800.33086511060
0.27091.249868500.31318278928
0.24221.499782200.29449895248
0.22641.749795900.270711490512
0.2351.9996109600.255413071912
0.16852.2496123300.260614810372
0.15382.4995137000.251916231800
0.15662.7495150700.245917815564
0.15482.9995164400.238719581608
0.11823.2494178100.259921161040
0.11063.4994191800.257622698788
0.11763.7493205500.255024438800
0.11753.9993219200.255826121308
0.10324.2492232900.264527832740
0.09984.4992246600.265629363700
0.1024.7491260300.265831053092
0.10194.9991274000.265932672364

Framework versions

  • —Transformers 4.40.2
  • —Pytorch 2.2.2+cu121
  • —Datasets 2.19.0
  • —Tokenizers 0.19.1