flytech/togetherchat-dev-7b-v2

1---2license: llama23base_model: togethercomputer/LLaMA-2-7B-32K4tags:5- generated_from_trainer6model-index:7- name: togetherchat-dev-7b-v28  results: []9---10 11<!-- This model card has been generated automatically according to the information the Trainer had access to. You12should probably proofread and complete it, then remove this comment. -->13 14# togetherchat-dev-7b-v215 16This model is a fine-tuned version of [togethercomputer/LLaMA-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K) on 25000 entries for 3 epochs.17 18## Model description19 20Model can be used for text-to-code generation and for further fine-tuning,21Colab notebook example (on free T4 GPU) soon!22 23## Datasets used:24 25- evol-codealpaca-80k - 10000 entries26- codealpaca-20k - 10000 entries27- open-platypus - 5000 entries28 29## Intended uses & limitations30 31Please remember that model may (and will) produce inaccurate informations, 32you need to fine-tune it for your specific task.33 34## Training and evaluation data35 36See 'Metrics'37 38### Training hyperparameters39 40The following hyperparameters were used during training:41- learning_rate: 0.000142- train_batch_size: 1043- eval_batch_size: 844- seed: 4245- gradient_accumulation_steps: 446- total_train_batch_size: 4047- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0848- lr_scheduler_type: linear49- lr_scheduler_warmup_ratio: 0.150- num_epochs: 351 52### Training results53 54 55 56### Framework versions57 58- Transformers 4.33.159- Pytorch 2.0.1+cu11860- Datasets 2.14.561- Tokenizers 0.13.362