AdinaM/hackathon_datacrowd

1---2language:3- fr4- it5- de6- es7- en8license: apache-2.09tags:10- moe11model-index:12- name: Mixtral-8x22B-v0.113  results:14  - task:15      type: text-generation16      name: Text Generation17    dataset:18      name: AI2 Reasoning Challenge (25-Shot)19      type: ai2_arc20      config: ARC-Challenge21      split: test22      args:23        num_few_shot: 2524    metrics:25    - type: acc_norm26      value: 70.4827      name: normalized accuracy28    source:29      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.130      name: Open LLM Leaderboard31  - task:32      type: text-generation33      name: Text Generation34    dataset:35      name: HellaSwag (10-Shot)36      type: hellaswag37      split: validation38      args:39        num_few_shot: 1040    metrics:41    - type: acc_norm42      value: 88.7343      name: normalized accuracy44    source:45      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.146      name: Open LLM Leaderboard47  - task:48      type: text-generation49      name: Text Generation50    dataset:51      name: MMLU (5-Shot)52      type: cais/mmlu53      config: all54      split: test55      args:56        num_few_shot: 557    metrics:58    - type: acc59      value: 77.8160      name: accuracy61    source:62      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.163      name: Open LLM Leaderboard64  - task:65      type: text-generation66      name: Text Generation67    dataset:68      name: TruthfulQA (0-shot)69      type: truthful_qa70      config: multiple_choice71      split: validation72      args:73        num_few_shot: 074    metrics:75    - type: mc276      value: 51.0877    source:78      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.179      name: Open LLM Leaderboard80  - task:81      type: text-generation82      name: Text Generation83    dataset:84      name: Winogrande (5-shot)85      type: winogrande86      config: winogrande_xl87      split: validation88      args:89        num_few_shot: 590    metrics:91    - type: acc92      value: 84.5393      name: accuracy94    source:95      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.196      name: Open LLM Leaderboard97  - task:98      type: text-generation99      name: Text Generation100    dataset:101      name: GSM8k (5-shot)102      type: gsm8k103      config: main104      split: test105      args:106        num_few_shot: 5107    metrics:108    - type: acc109      value: 74.15110      name: accuracy111    source:112      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=mistral-community/Mixtral-8x22B-v0.1113      name: Open LLM Leaderboard114---115# Mixtral-8x22B116 117> [!TIP]118> MistralAI has uploaded weights to their organization at [mistralai/Mixtral-8x22B-v0.1](https://huggingface.co/mistralai/Mixtral-8x22B-v0.1) and [mistralai/Mixtral-8x22B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1) too. 119 120> [!TIP]121> Kudos to [@v2ray](https://huggingface.co/v2ray) for converting the checkpoints and uploading them in `transformers` compatible format. Go give them a follow!122 123Converted to HuggingFace Transformers format using the script [here](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1/blob/main/convert.py).124 125The Mixtral-8x22B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts.126## Run the model127```python128from transformers import AutoModelForCausalLM, AutoTokenizer129 130model_id = "mistral-community/Mixtral-8x22B-v0.1"131tokenizer = AutoTokenizer.from_pretrained(model_id)132 133model = AutoModelForCausalLM.from_pretrained(model_id)134 135text = "Hello my name is"136inputs = tokenizer(text, return_tensors="pt")137 138outputs = model.generate(**inputs, max_new_tokens=20)139print(tokenizer.decode(outputs[0], skip_special_tokens=True))140```141By default, transformers will load the model in full precision. Therefore you might be interested to further reduce down the memory requirements to run the model through the optimizations we offer in HF ecosystem:142### In half-precision143Note `float16` precision only works on GPU devices144<details>145<summary> Click to expand </summary>146 147```diff148+ import torch149from transformers import AutoModelForCausalLM, AutoTokenizer150 151model_id = "mistral-community/Mixtral-8x22B-v0.1"152tokenizer = AutoTokenizer.from_pretrained(model_id)153 154+ model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16).to(0)155 156text = "Hello my name is"157+ inputs = tokenizer(text, return_tensors="pt").to(0)158 159outputs = model.generate(**inputs, max_new_tokens=20)160print(tokenizer.decode(outputs[0], skip_special_tokens=True))161```162</details>163 164### Lower precision using (8-bit & 4-bit) using `bitsandbytes`165<details>166<summary> Click to expand </summary>167 168```diff169+ import torch170from transformers import AutoModelForCausalLM, AutoTokenizer171 172model_id = "mistral-community/Mixtral-8x22B-v0.1"173tokenizer = AutoTokenizer.from_pretrained(model_id)174 175+ model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)176 177text = "Hello my name is"178+ inputs = tokenizer(text, return_tensors="pt").to(0)179 180outputs = model.generate(**inputs, max_new_tokens=20)181print(tokenizer.decode(outputs[0], skip_special_tokens=True))182```183</details>184 185### Load the model with Flash Attention 2186<details>187<summary> Click to expand </summary>188 189```diff190+ import torch191from transformers import AutoModelForCausalLM, AutoTokenizer192 193model_id = "mistral-community/Mixtral-8x22B-v0.1"194tokenizer = AutoTokenizer.from_pretrained(model_id)195 196+ model = AutoModelForCausalLM.from_pretrained(model_id, use_flash_attention_2=True)197 198text = "Hello my name is"199+ inputs = tokenizer(text, return_tensors="pt").to(0)200 201outputs = model.generate(**inputs, max_new_tokens=20)202print(tokenizer.decode(outputs[0], skip_special_tokens=True))203```204</details>205 206## Notice207Mixtral-8x22B-v0.1 is a pretrained base model and therefore does not have any moderation mechanisms.208# The Mistral AI Team209Albert Jiang, Alexandre Sablayrolles, Alexis Tacnet, Antoine Roux, Arthur Mensch, Audrey Herblin-Stoop, Baptiste Bout, Baudouin de Monicault,Blanche Savary, Bam4d, Caroline Feldman, Devendra Singh Chaplot, Diego de las Casas, Eleonore Arcelin, Emma Bou Hanna, Etienne Metzger, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Harizo Rajaona, Jean-Malo Delignon, Jia Li, Justus Murke, Louis Martin, Louis Ternon, Lucile Saulnier, Lélio Renard Lavaud, Margaret Jennings, Marie Pellat, Marie Torelli, Marie-Anne Lachaux, Nicolas Schuhl, Patrick von Platen, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Thibaut Lavril, Timothée Lacroix, Théophile Gervet, Thomas Wang, Valera Nemychnikova, William El Sayed, William Marshall.210# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)211Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_mistral-community__Mixtral-8x22B-v0.1)212 213|             Metric              |Value|214|---------------------------------|----:|215|Avg.                             |74.46|216|AI2 Reasoning Challenge (25-Shot)|70.48|217|HellaSwag (10-Shot)              |88.73|218|MMLU (5-Shot)                    |77.81|219|TruthfulQA (0-shot)              |51.08|220|Winogrande (5-shot)              |84.53|221|GSM8k (5-shot)                   |74.15|222 223