kaizerBox/ReFormer-small-summarization

1---2tags:3- generated_from_trainer4datasets:5- xsum6model-index:7- name: ReFormer-small-summarization8  results: []9---10 11<!-- This model card has been generated automatically according to the information the Trainer had access to. You12should probably proofread and complete it, then remove this comment. -->13 14# ReFormer-small-summarization15 16This model is a fine-tuned version of [](https://huggingface.co/) on the xsum dataset.17It achieves the following results on the evaluation set:18- Loss: 4.920219 20## Model description21 22More information needed23 24## Intended uses & limitations25 26More information needed27 28## Training and evaluation data29 30More information needed31 32## Training procedure33 34### Training hyperparameters35 36The following hyperparameters were used during training:37- learning_rate: 0.00138- train_batch_size: 839- eval_batch_size: 840- seed: 4241- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0842- lr_scheduler_type: cosine43- lr_scheduler_warmup_steps: 10044- num_epochs: 345- mixed_precision_training: Native AMP46 47### Training results48 49| Training Loss | Epoch | Step  | Validation Loss |50|:-------------:|:-----:|:-----:|:---------------:|51| 5.603         | 1.0   | 23050 | 5.1617          |52| 5.0493        | 2.0   | 46100 | 4.9603          |53| 4.9307        | 3.0   | 69150 | 4.9202          |54 55 56### Framework versions57 58- Transformers 4.35.259- Pytorch 2.1.0+cu11860- Datasets 2.15.061- Tokenizers 0.15.062