ajibawa-2023/Python-Code-33B

1---2language:3- en4license: cc-by-nc-nd-4.05tags:6- code7datasets:8- ajibawa-2023/Python-Code-23k-ShareGPT9model-index:10- name: Python-Code-33B11  results:12  - task:13      type: text-generation14      name: Text Generation15    dataset:16      name: AI2 Reasoning Challenge (25-Shot)17      type: ai2_arc18      config: ARC-Challenge19      split: test20      args:21        num_few_shot: 2522    metrics:23    - type: acc_norm24      value: 56.3125      name: normalized accuracy26    source:27      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B28      name: Open LLM Leaderboard29  - task:30      type: text-generation31      name: Text Generation32    dataset:33      name: HellaSwag (10-Shot)34      type: hellaswag35      split: validation36      args:37        num_few_shot: 1038    metrics:39    - type: acc_norm40      value: 81.0141      name: normalized accuracy42    source:43      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B44      name: Open LLM Leaderboard45  - task:46      type: text-generation47      name: Text Generation48    dataset:49      name: MMLU (5-Shot)50      type: cais/mmlu51      config: all52      split: test53      args:54        num_few_shot: 555    metrics:56    - type: acc57      value: 54.2258      name: accuracy59    source:60      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B61      name: Open LLM Leaderboard62  - task:63      type: text-generation64      name: Text Generation65    dataset:66      name: TruthfulQA (0-shot)67      type: truthful_qa68      config: multiple_choice69      split: validation70      args:71        num_few_shot: 072    metrics:73    - type: mc274      value: 44.3975    source:76      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B77      name: Open LLM Leaderboard78  - task:79      type: text-generation80      name: Text Generation81    dataset:82      name: Winogrande (5-shot)83      type: winogrande84      config: winogrande_xl85      split: validation86      args:87        num_few_shot: 588    metrics:89    - type: acc90      value: 75.2291      name: accuracy92    source:93      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B94      name: Open LLM Leaderboard95  - task:96      type: text-generation97      name: Text Generation98    dataset:99      name: GSM8k (5-shot)100      type: gsm8k101      config: main102      split: test103      args:104        num_few_shot: 5105    metrics:106    - type: acc107      value: 19.18108      name: accuracy109    source:110      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ajibawa-2023/Python-Code-33B111      name: Open LLM Leaderboard112---113 114**Python-Code-33B**115 116Large Language Models (LLMs) are good with code generations. Sometimes LLMs do make mistakes in code generation. How about if they can give detailed explanation along with the code.117This is what I have tried over here. The base Llama-2 model was used for training purpose. It is trained on around 23000+ set of codes. Each set having 2 conversations.118This data was generated using GPT-3.5, GPT-4 etc. This conversation is in Vicuna/ShareGPT format. Each set, along with code, has detailed explanation. 119I have released the [data](https://huggingface.co/datasets/ajibawa-2023/Python-Code-23k-ShareGPT).120 121**Training:**122Entire dataset was trained on Azure 4 x A100 80GB. For 3 epoch, training took 42 hours. DeepSpeed codebase was used for training purpose. This was trained on Llama-1 by Meta.123 124This is a full fine tuned model. Links for quantized models are given below.125 126 127**GPTQ GGML & AWQ**128 129GPTQ: [Link](https://huggingface.co/TheBloke/Python-Code-33B-GPTQ)130 131GGUF: [Link](https://huggingface.co/TheBloke/Python-Code-33B-GGUF)132 133AWQ: [Link](https://huggingface.co/TheBloke/Python-Code-33B-AWQ)134 135 136**Example Prompt:**137```138This is a conversation with your helpful AI assistant. AI assistant can generate Python Code along with necessary explanation.139 140Context141You are a helpful AI assistant.142 143USER: <prompt>144ASSISTANT:145```146# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)147Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ajibawa-2023__Python-Code-33B)148 149|             Metric              |Value|150|---------------------------------|----:|151|Avg.                             |55.06|152|AI2 Reasoning Challenge (25-Shot)|56.31|153|HellaSwag (10-Shot)              |81.01|154|MMLU (5-Shot)                    |54.22|155|TruthfulQA (0-shot)              |44.39|156|Winogrande (5-shot)              |75.22|157|GSM8k (5-shot)                   |19.18|158 159