Sharman16/roberta-context-dependency

1---2library_name: transformers3license: mit4base_model: roberta-base5tags:6- generated_from_trainer7metrics:8- accuracy9model-index:10- name: roberta-context-dependency11  results: []12---13 14<!-- This model card has been generated automatically according to the information the Trainer had access to. You15should probably proofread and complete it, then remove this comment. -->16 17# roberta-context-dependency18 19This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the None dataset.20It achieves the following results on the evaluation set:21- Loss: 0.830022- Accuracy: 0.615223- Macro F1: 0.558124- Weighted F1: 0.602825- Micro F1: 0.615226 27## Model description28 29More information needed30 31## Intended uses & limitations32 33More information needed34 35## Training and evaluation data36 37More information needed38 39## Training procedure40 41### Training hyperparameters42 43The following hyperparameters were used during training:44- learning_rate: 2e-0545- train_batch_size: 1646- eval_batch_size: 3247- seed: 4248- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments49- lr_scheduler_type: linear50- num_epochs: 451- mixed_precision_training: Native AMP52 53### Training results54 55| Training Loss | Epoch | Step | Validation Loss | Accuracy | Macro F1 | Weighted F1 | Micro F1 |56|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:-----------:|:--------:|57| 0.9588        | 1.0   | 296  | 0.9300          | 0.5752   | 0.3679   | 0.4925      | 0.5752   |58| 0.8767        | 2.0   | 592  | 0.8810          | 0.6038   | 0.5153   | 0.5800      | 0.6038   |59| 0.8247        | 3.0   | 888  | 0.8173          | 0.6038   | 0.5250   | 0.5851      | 0.6038   |60| 0.7298        | 4.0   | 1184 | 0.8300          | 0.6152   | 0.5581   | 0.6028      | 0.6152   |61 62 63### Framework versions64 65- Transformers 5.15.166- Pytorch 2.11.0+cu12867- Datasets 5.0.168- Tokenizers 0.22.269