BabyLM-community/sun-baseline-small

1---2library_name: transformers3tags:4- generated_from_trainer5model-index:6- name: sun-baseline-small7  results: []8---9 10<!-- This model card has been generated automatically according to the information the Trainer had access to. You11should probably proofread and complete it, then remove this comment. -->12 13# sun-baseline-small14 15This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.16It achieves the following results on the evaluation set:17- Loss: 4.844218 19## Model description20 21More information needed22 23## Intended uses & limitations24 25More information needed26 27## Training and evaluation data28 29More information needed30 31## Training procedure32 33### Training hyperparameters34 35The following hyperparameters were used during training:36- learning_rate: 0.000137- train_batch_size: 838- eval_batch_size: 839- seed: 4240- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments41- lr_scheduler_type: linear42- num_epochs: 1043 44### Training results45 46| Training Loss | Epoch | Step  | Validation Loss |47|:-------------:|:-----:|:-----:|:---------------:|48| 6.7143        | 1.0   | 1646  | 6.2067          |49| 5.9487        | 2.0   | 3292  | 5.7755          |50| 5.5457        | 3.0   | 4938  | 5.4530          |51| 5.2146        | 4.0   | 6584  | 5.2281          |52| 4.9624        | 5.0   | 8230  | 5.0784          |53| 4.7677        | 6.0   | 9876  | 4.9795          |54| 4.6107        | 7.0   | 11522 | 4.9167          |55| 4.4855        | 8.0   | 13168 | 4.8730          |56| 4.3889        | 9.0   | 14814 | 4.8511          |57| 4.3198        | 10.0  | 16460 | 4.8442          |58 59 60### Framework versions61 62- Transformers 4.53.063- Pytorch 2.7.1+cu12664- Datasets 4.0.065- Tokenizers 0.21.266