Galatea007/llama381binstruct_summarize_short

1---2base_model: NousResearch/Meta-Llama-3.1-8B-Instruct3datasets:4- generator5library_name: peft6license: llama3.17tags:8- trl9- sft10- generated_from_trainer11model-index:12- name: llama381binstruct_summarize_short13  results: []14---15 16<!-- This model card has been generated automatically according to the information the Trainer had access to. You17should probably proofread and complete it, then remove this comment. -->18 19# llama381binstruct_summarize_short20 21This model is a fine-tuned version of [NousResearch/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3.1-8B-Instruct) on the generator dataset.22It achieves the following results on the evaluation set:23- Loss: 2.796124 25## Model description26 27More information needed28 29## Intended uses & limitations30 31More information needed32 33## Training and evaluation data34 35More information needed36 37## Training procedure38 39### Training hyperparameters40 41The following hyperparameters were used during training:42- learning_rate: 0.000243- train_batch_size: 144- eval_batch_size: 845- seed: 4246- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0847- lr_scheduler_type: linear48- lr_scheduler_warmup_steps: 3049- training_steps: 50050 51### Training results52 53| Training Loss | Epoch | Step | Validation Loss |54|:-------------:|:-----:|:----:|:---------------:|55| 1.5831        | 1.25  | 25   | 1.6479          |56| 0.8086        | 2.5   | 50   | 1.6711          |57| 0.3333        | 3.75  | 75   | 1.8202          |58| 0.1834        | 5.0   | 100  | 1.9703          |59| 0.083         | 6.25  | 125  | 2.2035          |60| 0.0381        | 7.5   | 150  | 2.3948          |61| 0.0212        | 8.75  | 175  | 2.4440          |62| 0.0197        | 10.0  | 200  | 2.4377          |63| 0.0188        | 11.25 | 225  | 2.3988          |64| 0.0094        | 12.5  | 250  | 2.5389          |65| 0.0062        | 13.75 | 275  | 2.5661          |66| 0.0048        | 15.0  | 300  | 2.6391          |67| 0.0036        | 16.25 | 325  | 2.6828          |68| 0.0021        | 17.5  | 350  | 2.7307          |69| 0.0017        | 18.75 | 375  | 2.7560          |70| 0.0023        | 20.0  | 400  | 2.7722          |71| 0.0024        | 21.25 | 425  | 2.7839          |72| 0.002         | 22.5  | 450  | 2.7908          |73| 0.0021        | 23.75 | 475  | 2.7946          |74| 0.0018        | 25.0  | 500  | 2.7961          |75 76 77### Framework versions78 79- PEFT 0.12.080- Transformers 4.44.281- Pytorch 2.4.0+cu12182- Datasets 3.0.083- Tokenizers 0.19.1