jlpan/starcoder-finetuned-text2code

1---2license: bigcode-openrail-m3base_model: bigcode/starcoder4tags:5- generated_from_trainer6model-index:7- name: starcoder-finetuned-text2code8  results: []9---10 11<!-- This model card has been generated automatically according to the information the Trainer had access to. You12should probably proofread and complete it, then remove this comment. -->13 14# starcoder-finetuned-text2code15 16This model is a fine-tuned version of [bigcode/starcoder](https://huggingface.co/bigcode/starcoder) on an unknown dataset.17It achieves the following results on the evaluation set:18- Loss: 0.926719 20## Model description21 22More information needed23 24## Intended uses & limitations25 26More information needed27 28## Training and evaluation data29 30More information needed31 32## Training procedure33 34### Training hyperparameters35 36The following hyperparameters were used during training:37- learning_rate: 0.000138- train_batch_size: 139- eval_batch_size: 140- seed: 4241- gradient_accumulation_steps: 1642- total_train_batch_size: 1643- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0844- lr_scheduler_type: cosine45- lr_scheduler_warmup_steps: 10046- training_steps: 60047 48### Training results49 50| Training Loss | Epoch | Step | Validation Loss |51|:-------------:|:-----:|:----:|:---------------:|52| 1.1122        | 0.17  | 100  | 0.9820          |53| 0.8357        | 0.33  | 200  | 0.9464          |54| 0.7977        | 0.5   | 300  | 0.9342          |55| 0.7808        | 0.67  | 400  | 0.9294          |56| 0.7807        | 0.83  | 500  | 0.9268          |57| 0.7748        | 1.0   | 600  | 0.9267          |58 59 60### Framework versions61 62- Transformers 4.32.0.dev063- Pytorch 2.0.1+cu11764- Datasets 2.12.065- Tokenizers 0.13.366