datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mt_bench_prompts
MT Bench by LMSYS
This set of evaluation prompts is created by the LMSYS org for better evaluation of chat models.
For more information, see the paper.
Dataset loading
To load this dataset, use 🤗 datasets:
from datasets import load_dataset
data = load_dataset(HuggingFaceH4/mt_bench_prompts, split="train")
Dataset creation
To create the dataset, we do the following for our internal tooling.
rename turns to prompts,
add empty reference to remaining prompts… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/mt_bench_prompts.xcopa_mt XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
The Cross-lingual Choice of Plausible Alternatives dataset is a benchmark to evaluate the ability of machine learning models to transfer commonsense reasoning across
languages. The dataset is the translation and reannotation of the English COPA (Roemmele et al. 2011) and covers 11 languages from 11 families and several areas around
the globe. The dataset is challenging as it requires both the command of world knowledge and the ability to generalise to new languages. All the details about the
creation of XCOPA and the implementation of the baselines are available in the paper.\nAPIGen-MT-5k
Summary
APIGen-MT is an automated agentic data generation pipeline designed to synthesize verifiable, high-quality, realistic datasets for agentic applications
This dataset was released as part of APIGen-MT: Agentic PIpeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
Code: https://github.com/apigen-mt/apigen-mt.github.io
The repo contains 5000 multi-turn trajectories collected by APIGen-MT
This dataset is a subset of the data used to train the xLAM-2 model… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/APIGen-MT-5k.mt_bench_human_judgments
Content
This dataset contains 3.3K expert-level pairwise human preferences for model responses generated by 6 models in response to 80 MT-bench questions.
The 6 models are GPT-4, GPT-3.5, Claud-v1, Vicuna-13B, Alpaca-13B, and LLaMA-13B. The annotators are mostly graduate students with expertise in the topic areas of each of the questions. The details of data collection can be found in our paper.
Agreement Calculation
This Colab notebook shows how to compute the… See the full description on the dataset page: https://huggingface.co/datasets/lmsys/mt_bench_human_judgments.apex-r1-real-world-documents
Apex-R1 Real-World Benchmark Documents
This dataset stores real-world document/data assets collected for Apex-R1 synthetic long-horizon agentic RL workspace generation.
The files are intended as seed workspace materials, not as benchmark task labels. They can be injected into APEX-style filesystem/ or .apps_data/ environments to create more realistic and diverse professional-domain tasks.
Contents
benchmark_documents/
EnterpriseBench/ # CRM invoices… See the full description on the dataset page: https://huggingface.co/datasets/mtybilly/apex-r1-real-world-documents.MLQARetrieval
MLQARetrieval
An MTEB dataset
Massive Text Embedding Benchmark
MLQA (MultiLingual Question Answering) is a benchmark dataset for evaluating cross-lingual question answering performance.
MLQA consists of over 5K extractive QA instances (12K in English) in SQuAD format in seven languages - English, Arabic,
German, Spanish, Hindi, Vietnamese and Simplified Chinese. MLQA is highly parallel, with QA instances parallel between
4 different languages on average.… See the full description on the dataset page: https://huggingface.co/datasets/mteb/MLQARetrieval.GPT-4o-evaluation-biases
A database to support the evaluation of gender biases in GPT-4o output
The database and its construction process are described in the paper "A database to support the evaluation of gender biases in GPT-4o output" by Mehner et al., presented at the 1st ISCA/ITG Workshop on Diversity in Large Speech and Language Models (Berlin, Februar 20, 2025).
Introduction
This is a database of prompts and answers generated with GPT-4o-mini and GPT-4o in a pretest and a main test… See the full description on the dataset page: https://huggingface.co/datasets/mtec-TUB/GPT-4o-evaluation-biases.belebele
The Belebele Benchmark for Massively Multilingual NLU Evaluation
Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium-, and low-resource languages. Each question has four multiple-choice answers and is linked to a short passage from the FLORES-200 dataset. The human annotation procedure was carefully curated to create questions that discriminate… See the full description on the dataset page: https://huggingface.co/datasets/mteb/belebele.MCIF
Dataset Description, Collection, and Source
MCIF (Multimodal Crosslingual Instruction Following) is a multilingual human-annotated benchmark
based on scientific talks that is designed to evaluate instruction-following in crosslingual,
multimodal settings over both short- and long-form inputs.
MCIF spans three core modalities -- speech, vision, and text -- and four diverse languages (English, German, Italian, and Chinese),
enabling a comprehensive evaluation of MLLMs'… See the full description on the dataset page: https://huggingface.co/datasets/FBK-MT/MCIF.m_truthfulqa
Multilingual TruthfulQA
Dataset Summary
This dataset is a machine translated version of the TruthfulQA dataset, translated using GPT-3.5-turbo. This dataset was created by the University of Oregon, and was originally uploaded to this Github repository.
Citation
If you use this dataset in your work, please cite the following paper:
@article{dac2023okapi,
title={Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning… See the full description on the dataset page: https://huggingface.co/datasets/alexandrainst/m_truthfulqa.CQADupstackAndroidRetrieval
CQADupstackAndroidRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
CQADupStack: A Benchmark Data Set for Community Question-Answering Research
Task category
t2t
Domains
Programming, Web, Written, Non-fiction
Referencehttp://nlp.cis.unimelb.edu.au/resources/cqadupstack/
Source datasets:
mteb/cqadupstack-android
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CQADupstackAndroidRetrieval.WebFAQRetrieval
WebFAQRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
WebFAQ is a broad-coverage corpus of natural question-answer pairs in 75 languages, gathered from FAQ pages on the web.
Task category
t2t
Domains
Web, Written
Reference
https://huggingface.co/PaDaS-Lab
Source datasets:
PaDaS-Lab/webfaq-retrieval
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/WebFAQRetrieval.mt-bench-french
MT-Bench-French
This is a French version of MT-Bench, created to evaluate the multi-turn conversation and instruction-following capabilities of LLMs. Similar to its original version, MT-Bench-French comprises 80 high-quality, multi-turn questions spanning eight main categories.
All questions have undergone translation into French and thorough human review to guarantee the use of suitable and authentic wording, meaningful content for assessing LLMs' capabilities in the French… See the full description on the dataset page: https://huggingface.co/datasets/bofenghuang/mt-bench-french.MT-SFT-ShareGPT
MT-SFT-ShareGPT
💻 Github Repo • 🤗 HuggingFace • 🤖 ModelScope
Introduction
Data has always been an important part of advancing large language models forward. Based on this, we have collected dozens of high-quality open source datasets from the open source community, with a total data volume of 20 M.
After some cleaning actions, we have open sourced a set of high-quality datasets for fine-tuning the instructions of the… See the full description on the dataset page: https://huggingface.co/datasets/thomas-yanxin/MT-SFT-ShareGPT.cmath-mtThis dataset is a machine translated version of weitianwen/cmath.
Translated using dataset-translator.
distill-gpt4-eng-chat
Description
Introducing dataset consisting of gpt4 answers to users requests. Queries were taken from allenai/WildChat-1M and causal-lm/instructions. Texts (requests and responses) were deleted in 3 cases:
either has non-english letters and special symbols
either has http-links
either has html blocks
either has perplexity more than 1.5*IQR + third quantile ( in some cases average perplexity value of sentences or maximum value was used )
svq
Simple Voice Questions
Simple Voice Questions (SVQ) is a set of short audio questions recorded in 26 locales across 17 languages under multiple audio conditions.
Data Collection
Speakers were presented with recording instructions specifying the recording environment and text query to be recorded.
They recorded using their own phones or tablets under four conditions:
clean: Record in quiet environment
background speech noise: Record while audio from sources like podcasts… See the full description on the dataset page: https://huggingface.co/datasets/mteb/svq.XQuADRetrieval
XQuADRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
XQuAD is a benchmark dataset for evaluating cross-lingual question answering performance. It is repurposed retrieving relevant context for each question.
Task category
t2t
Domains
Web, Written
Reference
https://huggingface.co/datasets/xquad
Source datasets:
google/xquad
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb… See the full description on the dataset page: https://huggingface.co/datasets/mteb/XQuADRetrieval.GermanDPR
GermanDPR
An MTEB dataset
Massive Text Embedding Benchmark
GermanDPR is a German Question Answering dataset for open-domain QA. It associates questions with a textual context containing the answer
Task category
t2t
Domains
Written, Non-fiction, Web
Reference
https://huggingface.co/datasets/deepset/germandpr
Source datasets:
deepset/germandpr
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:… See the full description on the dataset page: https://huggingface.co/datasets/mteb/GermanDPR.mteb-fr-retrieval-syntec-s2p
Syntec dataset for information retrieval
This dataset has been built from the Syntec Collective bargaining agreement. Its purpose is information retrieval.
Dataset Details
The dataset is rather small. It is intended to be used only as a test set, for fast evaluation of models.
It is split into 2 subsets :
queries : it features 100 manually created questions. Each question is mapped to the article that contains the answer.
documents : corresponds to the 90 articles from… See the full description on the dataset page: https://huggingface.co/datasets/lyon-nlp/mteb-fr-retrieval-syntec-s2p.vqa-2MTRAG_taskA_results
MTRAG Task A: Multi-Turn RAG Retrieval Results
Complete experimental results and datasets for Multi-Turn Retrieval-Augmented Generation (RAG) research.
📊 Overview
This repository contains:
Experimental Results: 20 hybrid retrieval configurations + baselines (8.3 GB)
Raw Data: Passages, queries, rewrites, conversations (2.5 GB)
Documentation: Statistical validation, analysis, LaTeX presentation
Configurations: Reproducible experiment configs
🏆 Key Results… See the full description on the dataset page: https://huggingface.co/datasets/vania-janet/MTRAG_taskA_results.Electrical-engineering
To the electrical engineering community
This dataset contains Q&A prompts about electrical engineering, Kicad's EDA software features and scripting console Python codes.
Authors
STEM.AI: stem.ai.mtl@gmail.comWilliam Harbec
mTRECQA
Dataset Description
mTRECQA originates from TREC-QA, which is created from the TREC 8 to TREC 13 QA tracks. TREC 8-12 constitutes the training set, while TREC 13 questions are set aside for development and testing.
The dataset has been translated into five European languages: French, German, Italian, Portuguese, and Spanish, as described in this paper: Datasets for Multilingual Answer Sentence Selection.
Splits:
For each language (English, French, German, Italian… See the full description on the dataset page: https://huggingface.co/datasets/matteogabburo/mTRECQA.MTCMBRepository: For more information about the dataset, please refer to the GitHub website. https://github.com/Wayyuanyuan/MTCMB/tree/main
Point of Contact: If you have any questions about this dataset, please contact my email: yangxingru2020@126.com
bigbenchhard-mt-pt
BBH-PT (Big-Bench Hard)
Portuguese machine translation of BIG-Bench Hard, a challenging subset of the BIG-Bench benchmark covering diverse reasoning tasks.
Translated using a Finetuned GemmaX2-9B for pt-PT with rule-based adaptations.
Note: Some tasks (e.g., hyperbaton) are not translated as they do not transfer meaningfully to Portuguese.
Original Dataset: https://github.com/suzgunmirac/BIG-Bench-Hard
Note: This dataset is machine translated and may contain… See the full description on the dataset page: https://huggingface.co/datasets/amalia-llm/bigbenchhard-mt-pt.tatoeba-mt-qna-oa
Dataset Card for multilingual tatoeba QnA translation with ~120K entries.
Dataset Summary
Contains Parquet of a list of instructions and translation articles on different languages.
Each row consists of
INSTRUCTION
RESPONSE
SOURCE (tatoeba)
METADATA (json with language, text length, uuid, langs-pair).
Original Dataset is avalible here:
https://huggingface.co/datasets/Helsinki-NLP/tatoeba_mt
JaqketRetrievalLite
JaqketRetrievalLite
An MTEB dataset
Massive Text Embedding Benchmark
JAQKET (JApanese Questions on Knowledge of EnTities) is a QA dataset created based on quiz questions. This is the lightweight version with a reduced corpus (65,802 documents) constructed using hard negatives from 5 high-performance models.
Task category
t2t
Domains
Encyclopaedic, Non-fiction, Written
Reference
https://github.com/kumapo/JAQKET-dataset
Source datasets:
sbintuitions/JMTEB-lite… See the full description on the dataset page: https://huggingface.co/datasets/mteb/JaqketRetrievalLite.MT-Video-Bench
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues
✨ Introduction
Recent advances in multimodal large language models (MLLMs) have brought remarkable progress in video understanding.However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios.
🎬 MT-Video-Bench fills this… See the full description on the dataset page: https://huggingface.co/datasets/NJU-LINK/MT-Video-Bench.apigen-mt-5k-parsed
[PARSED] APIGen-MT-5k
The data in this dataset is a full of the original Salesforce/APIGen-MT-5k
Subset name
multi-turn
parallel
multiple definition
Last turn type
number of dataset
apigen-mt-5k
yes
no
yes
complex
5k
This is a re-parsing formatting dataset for the APIGen-MT-5k official dataset.
Load the dataset
from datasets import load_dataset
ds = load_dataset("minpeter/apigen-mt-5k-parsed")
print(ds)
# DatasetDict({
# train: Dataset({
#… See the full description on the dataset page: https://huggingface.co/datasets/minpeter/apigen-mt-5k-parsed.