CharlesLi/llama_2_cot_simplest_code_math_1_full

1---2library_name: transformers3license: llama24base_model: meta-llama/Llama-2-7b-chat-hf5tags:6- alignment-handbook7- trl8- sft9- generated_from_trainer10- trl11- sft12- alignment-handbook13- generated_from_trainer14datasets:15- generator16model-index:17- name: llama_2_cot_simplest_code_math_1_full18  results: []19---20 21<!-- This model card has been generated automatically according to the information the Trainer had access to. You22should probably proofread and complete it, then remove this comment. -->23 24# llama_2_cot_simplest_code_math_1_full25 26This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on the generator dataset.27It achieves the following results on the evaluation set:28- Loss: 0.790229 30## Model description31 32More information needed33 34## Intended uses & limitations35 36More information needed37 38## Training and evaluation data39 40More information needed41 42## Training procedure43 44### Training hyperparameters45 46The following hyperparameters were used during training:47- learning_rate: 2e-0548- train_batch_size: 449- eval_batch_size: 450- seed: 4251- distributed_type: multi-GPU52- num_devices: 453- gradient_accumulation_steps: 254- total_train_batch_size: 3255- total_eval_batch_size: 1656- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0857- lr_scheduler_type: cosine58- lr_scheduler_warmup_ratio: 0.159- num_epochs: 160 61### Training results62 63 64 65### Framework versions66 67- Transformers 4.44.268- Pytorch 2.4.1+cu12169- Datasets 3.0.070- Tokenizers 0.19.171