Hartunka/tiny_bert_rand_5_v2_sst2

1---2library_name: transformers3language:4- en5base_model: Hartunka/tiny_bert_rand_5_v26tags:7- generated_from_trainer8datasets:9- glue10metrics:11- accuracy12model-index:13- name: tiny_bert_rand_5_v2_sst214  results:15  - task:16      name: Text Classification17      type: text-classification18    dataset:19      name: GLUE SST220      type: glue21      args: sst222    metrics:23    - name: Accuracy24      type: accuracy25      value: 0.816513761467889926---27 28<!-- This model card has been generated automatically according to the information the Trainer had access to. You29should probably proofread and complete it, then remove this comment. -->30 31# tiny_bert_rand_5_v2_sst232 33This model is a fine-tuned version of [Hartunka/tiny_bert_rand_5_v2](https://huggingface.co/Hartunka/tiny_bert_rand_5_v2) on the GLUE SST2 dataset.34It achieves the following results on the evaluation set:35- Loss: 0.448736- Accuracy: 0.816537 38## Model description39 40More information needed41 42## Intended uses & limitations43 44More information needed45 46## Training and evaluation data47 48More information needed49 50## Training procedure51 52### Training hyperparameters53 54The following hyperparameters were used during training:55- learning_rate: 5e-0556- train_batch_size: 25657- eval_batch_size: 25658- seed: 1059- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments60- lr_scheduler_type: linear61- num_epochs: 5062 63### Training results64 65| Training Loss | Epoch | Step | Validation Loss | Accuracy |66|:-------------:|:-----:|:----:|:---------------:|:--------:|67| 0.427         | 1.0   | 264  | 0.4487          | 0.8165   |68| 0.2402        | 2.0   | 528  | 0.5560          | 0.7947   |69| 0.1856        | 3.0   | 792  | 0.5291          | 0.8050   |70| 0.155         | 4.0   | 1056 | 0.5920          | 0.7959   |71| 0.1259        | 5.0   | 1320 | 0.6380          | 0.7936   |72| 0.1068        | 6.0   | 1584 | 0.7638          | 0.8016   |73 74 75### Framework versions76 77- Transformers 4.50.278- Pytorch 2.2.1+cu12179- Datasets 2.18.080- Tokenizers 0.21.181