CoolFace
Modelpublic

AlekseyKorshuk/twscrape-prepared-regression-e5-base-4k-3epochs

sourceHugging Facemitupdated 2y agoView on Hugging Face
0likes14downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

twscrape-prepared-regression-e5-base-4k-3epochs

This model is a fine-tuned version of dwzhu/e5-base-4k on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 1.4350
  • —Mse: 0.0003
  • —Target 0 Mse: 0.0009
  • —Target 0 Distributions: <wandb.sdk.data_types.image.Image object at 0x7f5b2a1ac610>
  • —Target 0 Error Distribution: <wandb.sdk.data_types.image.Image object at 0x7f5c314b43a0>
  • —Target 1 Mse: 0.0003
  • —Target 1 Distributions: <wandb.sdk.data_types.image.Image object at 0x7f5c3131fdf0>
  • —Target 1 Error Distribution: <wandb.sdk.data_types.image.Image object at 0x7f5c180ffaf0>
  • —Target 2 Mse: 0.0001
  • —Target 2 Distributions: <wandb.sdk.data_types.image.Image object at 0x7f5c1809f5e0>
  • —Target 2 Error Distribution: <wandb.sdk.data_types.image.Image object at 0x7f5c3015e920>
  • —Target 3 Mse: 0.0000
  • —Target 3 Distributions: <wandb.sdk.data_types.image.Image object at 0x7f5c312b04f0>
  • —Target 3 Error Distribution: <wandb.sdk.data_types.image.Image object at 0x7f5c312b3340>

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 1e-05
  • —trainbatchsize: 32
  • —evalbatchsize: 32
  • —seed: 42
  • —distributed_type: multi-GPU
  • —num_devices: 8
  • —totaltrainbatch_size: 256
  • —totalevalbatch_size: 256
  • —optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
  • —lrschedulertype: cosine
  • —lrschedulerwarmup_ratio: 0.1
  • —num_epochs: 3.0

Training results

Training LossEpochStepValidation LossMseTarget 0 MseTarget 0 DistributionsTarget 0 Error DistributionTarget 1 MseTarget 1 DistributionsTarget 1 Error DistributionTarget 2 MseTarget 2 DistributionsTarget 2 Error DistributionTarget 3 MseTarget 3 DistributionsTarget 3 Error Distribution
1.45611.015891.47440.00030.0010<wandb.sdk.data_types.image.Image object at 0x7f5c32c2e530><wandb.sdk.data_types.image.Image object at 0x7f5c32c2e620>0.0003<wandb.sdk.data_types.image.Image object at 0x7f5c4414f610><wandb.sdk.data_types.image.Image object at 0x7f5c473e3340>0.0001<wandb.sdk.data_types.image.Image object at 0x7f5c47230ac0><wandb.sdk.data_types.image.Image object at 0x7f5c4711f820>0.0000<wandb.sdk.data_types.image.Image object at 0x7f5c440fb250><wandb.sdk.data_types.image.Image object at 0x7f5c474deb60>
1.46482.031781.44010.00030.0009<wandb.sdk.data_types.image.Image object at 0x7f5c47483f40><wandb.sdk.data_types.image.Image object at 0x7f5c442efa30>0.0003<wandb.sdk.data_types.image.Image object at 0x7f5c3639dfc0><wandb.sdk.data_types.image.Image object at 0x7f5c1839de40>0.0001<wandb.sdk.data_types.image.Image object at 0x7f5c1843d9c0><wandb.sdk.data_types.image.Image object at 0x7f5c182ce5f0>0.0000<wandb.sdk.data_types.image.Image object at 0x7f5c3627e980><wandb.sdk.data_types.image.Image object at 0x7f5c36151f00>
1.16853.047671.43500.00030.0009<wandb.sdk.data_types.image.Image object at 0x7f5c32e8c2b0><wandb.sdk.data_types.image.Image object at 0x7f5c3610c1c0>0.0003<wandb.sdk.data_types.image.Image object at 0x7f5a8ea24640><wandb.sdk.data_types.image.Image object at 0x7f5c32defee0>0.0001<wandb.sdk.data_types.image.Image object at 0x7f5c315609a0><wandb.sdk.data_types.image.Image object at 0x7f5a8ea248b0>0.0000<wandb.sdk.data_types.image.Image object at 0x7f5b2a27d900><wandb.sdk.data_types.image.Image object at 0x7f5c316b6710>

Framework versions

  • —Transformers 4.49.0
  • —Pytorch 2.5.1+cu124
  • —Datasets 3.0.1
  • —Tokenizers 0.21.0