Dynosaur/llama3-8b-math-sft-subtask-8-subset

1---2library_name: transformers3license: llama34base_model: Dynosaur/llama3-8b-math-sft5tags:6- alignment-handbook7- trl8- sft9- generated_from_trainer10- trl11- sft12- generated_from_trainer13datasets:14- Dynosaur/math-sft-subtask-8-subset15model-index:16- name: llama3-8b-math-sft-subtask-8-subset17  results: []18---19 20<!-- This model card has been generated automatically according to the information the Trainer had access to. You21should probably proofread and complete it, then remove this comment. -->22 23# llama3-8b-math-sft-subtask-8-subset24 25This model is a fine-tuned version of [Dynosaur/llama3-8b-math-sft](https://huggingface.co/Dynosaur/llama3-8b-math-sft) on the Dynosaur/math-sft-subtask-8-subset dataset.26 27## Model description28 29More information needed30 31## Intended uses & limitations32 33More information needed34 35## Training and evaluation data36 37More information needed38 39## Training procedure40 41### Training hyperparameters42 43The following hyperparameters were used during training:44- learning_rate: 2e-0545- train_batch_size: 246- eval_batch_size: 847- seed: 4248- distributed_type: multi-GPU49- num_devices: 850- gradient_accumulation_steps: 851- total_train_batch_size: 12852- total_eval_batch_size: 6453- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0854- lr_scheduler_type: cosine55- lr_scheduler_warmup_ratio: 0.156- num_epochs: 257 58### Training results59 60 61 62### Framework versions63 64- Transformers 4.44.265- Pytorch 2.4.1+cu12166- Datasets 3.0.067- Tokenizers 0.19.168