CoolFace
Modelpublic

mrm8488/bloom-560m-finetuned-unnatural-instructions

sourceHugging Facebigscience-bloom-rail-1.0updated 4y agoView on Hugging Face
0likes26downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

bloom-560m-finetuned-unnatural-instructions

This model is a fine-tuned version of bigscience/bloom-560m on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 2.7021

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-05
  • —trainbatchsize: 1
  • —evalbatchsize: 2
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —num_epochs: 5
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
1.69370.3210001.6739
1.55270.6320001.5767
1.53050.9530001.5221
1.15141.2640001.5201
1.15641.5850001.5042
1.13651.8960001.4799
0.77292.2170001.6496
0.77132.5280001.5909
0.80632.8490001.6073
0.47533.15100001.9611
0.47193.47110002.0177
0.47323.79120002.0341
0.27474.1130002.5669
0.25824.42140002.6801
0.27514.73150002.6907

Framework versions

  • —Transformers 4.25.1
  • —Pytorch 1.13.0+cu116
  • —Datasets 2.8.0
  • —Tokenizers 0.13.2