HuggingFaceH4/mt_bench_prompts

1---2license: apache-2.03task_categories:4- question-answering5- conversational6language:7- en8tags:9- evaluation10pretty_name: MT Bench11size_categories:12- n<1K13---14 15# MT Bench by LMSYS16This set of evaluation prompts is created by the [LMSYS org](https://huggingface.co/lmsys) for better evaluation of chat models.17For more information, see the [paper](https://arxiv.org/abs/2306.05685).18 19 20### Dataset loading21To load this dataset, use 🤗 datasets:22```python23from datasets import load_dataset24data = load_dataset(HuggingFaceH4/mt_bench_prompts, split="train")25```26### Dataset creation27To create the dataset, we do the following for our internal tooling. 28* rename `turns` to `prompts`,29* add empty `reference` to remaining prompts (for HF Datasets),30* Use the following code to load and save as a dataset31```python32from datasets import load_dataset33import hashlib34 35data = load_dataset("json", data_files="https://huggingface.co/datasets/HuggingFaceH4/mt_bench_prompts/raw/main/raw/question.jsonl", split="train")36 37# %% create_dataset.ipynb 1138def format_example(example):39    return {40        "prompt": example["prompt"],41        "prompt_id": int(hashlib.sha256(''.join(example["prompt"]).encode("utf-8")).hexdigest(), 16) % (10 ** 8),42        "category": example["category"],43        "reference": example["reference"],44    }45 46formatted_ds = data.map(format_example, num_proc=6, remove_columns=data.column_names)47 48# 49formatted_ds.push_to_hub("HuggingFaceH4/mt_bench_prompts", split="train")50```