CoolFace
Modelpublic

joaomatos7/query

sourceHugging Facellama3updated 2y agoView on Hugging Face
0likes5downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

query

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.3182

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 2e-05
  • —trainbatchsize: 4
  • —evalbatchsize: 4
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —num_epochs: 32
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
No log1.0414.3920
No log2.0813.2479
No log3.01212.0852
No log4.01610.4945
No log5.0208.7942
No log6.0246.9466
No log7.0284.9444
No log8.0322.8411
No log9.0361.2720
No log10.0400.7259
No log11.0440.5651
No log12.0480.5014
No log13.0520.4775
No log14.0560.4608
No log15.0600.4456
No log16.0640.4314
No log17.0680.4171
No log18.0720.4032
No log19.0760.3910
No log20.0800.3808
No log21.0840.3719
No log22.0880.3638
No log23.0920.3561
No log24.0960.3491
No log25.01000.3426
No log26.01040.3370
No log27.01080.3320
No log28.01120.3278
No log29.01160.3241
No log30.01200.3216
No log31.01240.3195
No log32.01280.3182

Framework versions

  • —PEFT 0.10.0
  • —Transformers 4.41.1
  • —Pytorch 2.3.0+cu121
  • —Datasets 2.19.1
  • —Tokenizers 0.19.1