dugoalberto/starcoderbase_1b_LoRA

1---2license: bigcode-openrail-m3library_name: peft4tags:5- trl6- sft7- generated_from_trainer8base_model: bigcode/starcoderbase-1b9model-index:10- name: starcoderbase1_LoRA11  results: []12---13 14<!-- This model card has been generated automatically according to the information the Trainer had access to. You15should probably proofread and complete it, then remove this comment. -->16 17[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/albertodugo/huggingface/runs/luuyi9uk)18# starcoderbase1_LoRA19 20This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on an unknown dataset.21 22## Model description23 24More information needed25 26## Intended uses & limitations27 28More information needed29 30## Training and evaluation data31 32More information needed33 34## Training procedure35 36### Training hyperparameters37 38The following hyperparameters were used during training:39- learning_rate: 5e-0540- train_batch_size: 841- eval_batch_size: 842- seed: 4243- gradient_accumulation_steps: 444- total_train_batch_size: 3245- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0846- lr_scheduler_type: cosine47- lr_scheduler_warmup_ratio: 0.0548- training_steps: 6049- mixed_precision_training: Native AMP50 51### Training results52 53 54 55### Framework versions56 57- PEFT 0.11.2.dev058- Transformers 4.41.059- Pytorch 2.2.1+cu12160- Datasets 2.19.161- Tokenizers 0.19.1