rombodawg/Everyone-Coder-33b-Base

1---2license: other3tags:4- merge5license_name: deepseek6license_link: https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL7model-index:8- name: Everyone-Coder-33b-Base9  results:10  - task:11      type: text-generation12      name: Text Generation13    dataset:14      name: AI2 Reasoning Challenge (25-Shot)15      type: ai2_arc16      config: ARC-Challenge17      split: test18      args:19        num_few_shot: 2520    metrics:21    - type: acc_norm22      value: 45.9923      name: normalized accuracy24    source:25      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base26      name: Open LLM Leaderboard27  - task:28      type: text-generation29      name: Text Generation30    dataset:31      name: HellaSwag (10-Shot)32      type: hellaswag33      split: validation34      args:35        num_few_shot: 1036    metrics:37    - type: acc_norm38      value: 61.7139      name: normalized accuracy40    source:41      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base42      name: Open LLM Leaderboard43  - task:44      type: text-generation45      name: Text Generation46    dataset:47      name: MMLU (5-Shot)48      type: cais/mmlu49      config: all50      split: test51      args:52        num_few_shot: 553    metrics:54    - type: acc55      value: 44.0556      name: accuracy57    source:58      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base59      name: Open LLM Leaderboard60  - task:61      type: text-generation62      name: Text Generation63    dataset:64      name: TruthfulQA (0-shot)65      type: truthful_qa66      config: multiple_choice67      split: validation68      args:69        num_few_shot: 070    metrics:71    - type: mc272      value: 42.2673    source:74      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base75      name: Open LLM Leaderboard76  - task:77      type: text-generation78      name: Text Generation79    dataset:80      name: Winogrande (5-shot)81      type: winogrande82      config: winogrande_xl83      split: validation84      args:85        num_few_shot: 586    metrics:87    - type: acc88      value: 63.0689      name: accuracy90    source:91      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base92      name: Open LLM Leaderboard93  - task:94      type: text-generation95      name: Text Generation96    dataset:97      name: GSM8k (5-shot)98      type: gsm8k99      config: main100      split: test101      args:102        num_few_shot: 5103    metrics:104    - type: acc105      value: 39.8106      name: accuracy107    source:108      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=rombodawg/Everyone-Coder-33b-Base109      name: Open LLM Leaderboard110---111Everyone-Coder-33b-Base112 113 114![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/ECrHQnZnv8UM9GUCQtlWW.jpeg)115 116EveryoneLLM series of models made by the community, for the community. This is a coding specific model made using fine-tunes of deekseekcoder-33b-base.117 118______________________________________________________________________________________________________________119Im having trouble benchmarking this model because I suck at running llm benchmarks, but from hand testing running the model through https://edabit.com/challenge coding challenges vs up to date gpt-4. My model is hands down beating it in coding. 120______________________________________________________________________________________________________________121Ive recently noticed this model has trouble with end tokens so I made a custom prompt template for it. Made sure to add (Always end with "<|EOT|>") In addition to your system prompt and  (Always end your response with "<|EOT|>") at the end of the User message is the preset. Then add <|EOT|> as a custom stop string in your LM text generating interface. 122 123```124Always end with "<|EOT|>"125 126{System}127 128<|User|>129 130{User}. Always end your response with "<|EOT|>"131 132<|Assistant|>133 134{Assistant}135```136 137The models that were used in this merger were as follow:138 139- https://huggingface.co/deepseek-ai/deepseek-coder-33b-instruct140 141- https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B142 143- https://huggingface.co/WizardLM/WizardCoder-33B-V1.1144 145Thank you to the creators of the above ai models, they have full credit for the EveryoneLLM series of models. Without their hard work we wouldnt be able to achieve the great success we have in the open source community. 💗146 147You can find the write up for merging models here:148 149https://docs.google.com/document/d/1_vOftBnrk9NRk5h10UqrfJ5CDih9KBKL61yvrZtVWPE/edit?usp=sharing150 151Config for the merger can be found bellow:152 153```yaml154models:155  - model: WizardLM_WizardCoder-33B-V1.1156    parameters:157      density: 1158      weight: .5159  - model: codefuse-ai_CodeFuse-DeepSeek-33B160    parameters:161      density: 1162      weight: .5163merge_method: ties164base_model: deepseek-ai_deepseek-coder-33b-instruct165parameters:166  normalize: true167  int8_mask: true168dtype: float16169 170```171 172# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)173Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_rombodawg__Everyone-Coder-33b-Base)174 175|             Metric              |Value|176|---------------------------------|----:|177|Avg.                             |49.48|178|AI2 Reasoning Challenge (25-Shot)|45.99|179|HellaSwag (10-Shot)              |61.71|180|MMLU (5-Shot)                    |44.05|181|TruthfulQA (0-shot)              |42.26|182|Winogrande (5-shot)              |63.06|183|GSM8k (5-shot)                   |39.80|184 185