CoolFace
Modelpublic

amazon-sagemaker-community/xlm-roberta-en-ru-emoji-v2

sourceHugging Faceupdated 3y agoView on Hugging Face
1likes29downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

xlm-roberta-en-ru-emoji-v2

This model is a fine-tuned version of DeepPavlov/xlm-roberta-large-en-ru on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 2.3356
  • —Accuracy: 0.3102

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 2e-05
  • —trainbatchsize: 96
  • —evalbatchsize: 96
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_steps: 500
  • —num_epochs: 5
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation LossAccuracy
No log0.42003.05920.1204
No log0.814002.53560.2480
2.62941.216002.45700.2569
2.62941.628002.33320.2832
1.92862.0210002.33540.2803
1.92862.4212002.36100.2881
1.92862.8314002.30040.2973
1.73123.2316002.36190.3026
1.73123.6418002.35960.3032
1.58164.0420002.29720.3072
1.58164.4422002.30770.3073
1.58164.8524002.33560.3102

Framework versions

  • —Transformers 4.12.3
  • —Pytorch 1.9.1
  • —Datasets 1.15.1
  • —Tokenizers 0.10.3