flyover19/santacoder-finetuned-the-stack-bash

1---2license: bigcode-openrail-m3base_model: bigcode/santacoder4tags:5- generated_from_trainer6model-index:7- name: santacoder-finetuned-the-stack-bash8  results: []9---10 11<!-- This model card has been generated automatically according to the information the Trainer had access to. You12should probably proofread and complete it, then remove this comment. -->13 14# santacoder-finetuned-the-stack-bash15 16This model is a fine-tuned version of [bigcode/santacoder](https://huggingface.co/bigcode/santacoder) on an unknown dataset.17It achieves the following results on the evaluation set:18- Loss: 0.220219 20## Model description21 22More information needed23 24## Intended uses & limitations25 26More information needed27 28## Training and evaluation data29 30More information needed31 32## Training procedure33 34### Training hyperparameters35 36The following hyperparameters were used during training:37- learning_rate: 5e-0538- train_batch_size: 139- eval_batch_size: 140- seed: 4241- gradient_accumulation_steps: 442- total_train_batch_size: 443- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0844- lr_scheduler_type: cosine45- lr_scheduler_warmup_steps: 10046- training_steps: 500047 48### Training results49 50| Training Loss | Epoch | Step | Validation Loss |51|:-------------:|:-----:|:----:|:---------------:|52| 1.7564        | 0.1   | 500  | 1.3213          |53| 1.6757        | 0.2   | 1000 | 4.5570          |54| 1.6668        | 0.3   | 1500 | 7.4934          |55| 0.4505        | 0.4   | 2000 | 0.4260          |56| 1.6604        | 0.5   | 2500 | 0.5150          |57| 1.6552        | 0.6   | 3000 | 0.5775          |58| 1.6481        | 0.7   | 3500 | 0.6173          |59| 1.656         | 0.8   | 4000 | 0.2171          |60| 1.6554        | 0.9   | 4500 | 0.2198          |61| 1.6563        | 1.0   | 5000 | 0.2202          |62 63 64### Framework versions65 66- Transformers 4.33.367- Pytorch 2.0.168- Datasets 2.14.569- Tokenizers 0.13.370