S2312dal/M5_MLM

1---2license: mit3tags:4- generated_from_trainer5model-index:6- name: M5_MLM7  results: []8---9 10<!-- This model card has been generated automatically according to the information the Trainer had access to. You11should probably proofread and complete it, then remove this comment. -->12 13# M5_MLM14 15This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on an unknown dataset.16It achieves the following results on the evaluation set:17- Loss: 7.044718 19## Model description20 21More information needed22 23## Intended uses & limitations24 25More information needed26 27## Training and evaluation data28 29More information needed30 31## Training procedure32 33### Training hyperparameters34 35The following hyperparameters were used during training:36- learning_rate: 2e-0537- train_batch_size: 638- eval_batch_size: 639- seed: 4240- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0841- lr_scheduler_type: linear42- num_epochs: 3.043- mixed_precision_training: Native AMP44 45### Training results46 47| Training Loss | Epoch | Step | Validation Loss |48|:-------------:|:-----:|:----:|:---------------:|49| 9.8279        | 1.0   | 62   | 7.9889          |50| 7.7536        | 2.0   | 124  | 7.3750          |51| 7.2065        | 3.0   | 186  | 6.8625          |52 53 54### Framework versions55 56- Transformers 4.20.057- Pytorch 1.11.0+cu11358- Datasets 2.3.259- Tokenizers 0.12.160