ViktorDo/DistilBERT-POWO_Life_Form_Finetuned

1---2license: apache-2.03tags:4- generated_from_trainer5model-index:6- name: DistilBERT-POWO_Life_Form_Finetuned7  results: []8---9 10<!-- This model card has been generated automatically according to the information the Trainer had access to. You11should probably proofread and complete it, then remove this comment. -->12 13# DistilBERT-POWO_Life_Form_Finetuned14 15This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.16It achieves the following results on the evaluation set:17- Loss: 0.406318 19## Model description20 21More information needed22 23## Intended uses & limitations24 25More information needed26 27## Training and evaluation data28 29More information needed30 31## Training procedure32 33### Training hyperparameters34 35The following hyperparameters were used during training:36- learning_rate: 2e-0537- train_batch_size: 1638- eval_batch_size: 1639- seed: 4240- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0841- lr_scheduler_type: linear42- num_epochs: 343- mixed_precision_training: Native AMP44 45### Training results46 47| Training Loss | Epoch | Step | Validation Loss |48|:-------------:|:-----:|:----:|:---------------:|49| 0.4726        | 1.0   | 1004 | 0.4425          |50| 0.3852        | 2.0   | 2008 | 0.4087          |51| 0.3397        | 3.0   | 3012 | 0.4063          |52 53 54### Framework versions55 56- Transformers 4.23.157- Pytorch 1.12.1+cu11358- Datasets 2.6.159- Tokenizers 0.13.160