datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
IfEvalCode-testsetpreference-test-sets
Preference Test Sets
Very few preference datasets have heldout test sets for validation of reward model accuracy results.
In this dataset, we curate the test sets from popular preference datasets into a common schema for easy loading and evaluation.
Anthropic HH (Helpful & Harmless Agent and Red Teaming), test set in full is 8552 samples
Anthropic HHH Alignment (Helpful, Honest, & Harmless), formatted from Big Bench for standalone evaluation.
Learning to summarize, downsampled from… See the full description on the dataset page: https://huggingface.co/datasets/allenai/preference-test-sets.SpatialLM-Testset
SpatialLM Testset
Project page | Paper | Code
We provide a test set of 107 preprocessed point clouds and their corresponding GT layouts, point clouds are reconstructed from RGB videos using MASt3R-SLAM. SpatialLM-Testset is quite challenging compared to prior clean RGBD scan datasets due to the noises and occlusions in the point clouds reconstructed from monocular RGB videos.
Folder Structure
Outlines of the dataset files:… See the full description on the dataset page: https://huggingface.co/datasets/manycore-research/SpatialLM-Testset.TTS-Multilingual-Test-Set
Overview
To assess the multilingual zero-shot voice cloning capabilities of TTS models, we have constructed a test set encompassing 24 languages. This dataset provides both audio samples for voice cloning and corresponding test texts.
Specifically, the test set for each language includes:
100 distinct test sentences.
Audio samples from two speakers (one male and one female) carefully selected from the Mozilla Common Voice (MCV) dataset, intended for voice cloning.
Researchers can… See the full description on the dataset page: https://huggingface.co/datasets/MiniMaxAI/TTS-Multilingual-Test-Set.emu_edit_test_set
Dataset Card for the Emu Edit Test Set
Dataset Summary
To create a benchmark for image editing we first define seven different categories of potential image editing operations: background alteration (background), comprehensive image changes (global), style alteration (style), object removal (remove), object addition (add), localized modifications (local), and color/texture alterations (texture).
Then, we utilize the diverse set of input images from the MagicBrush… See the full description on the dataset page: https://huggingface.co/datasets/facebook/emu_edit_test_set.video-SALMONN_2_testset
video-SALMONN 2 Benchmark
Generate the caption corresponding to the video and the audio with video_salmonn2_test.json
Organize your results in the format like the following example:
[
{
"id": ["0.mp4"],
"pred": "Generated Caption"
}
]
Replace res_file in eval.py with your result file.
Run python3 eval.pytestset_piqatestset_popqaindexed-open-image-v4-test-set
Dataset Card for "indexed-open-image-v4-test-set"
More Information needed
MindGuard-testset
MindGuard-testset: Expert-Annotated Evaluation Data for Mental Health AI Safety
MindGuard-testset is a clinically grounded benchmark dataset for evaluating safety classifiers in mental health AI systems. This dataset was developed by Sword Health in collaboration with licensed clinical psychologists to address the critical need for contextually appropriate safety measures in therapeutic AI applications.
Overview
MindGuard-testset contains 1,134 annotated user turns… See the full description on the dataset page: https://huggingface.co/datasets/swordhealth/MindGuard-testset.vindr-cxr-testsettestset_mmlutestset_hellaswagtestset_winogrande-infillWorldRenderer-TestsetMMM-datasets-TestsetMultilingual Mutual Reinforcement Effect Mix Datasets
This is a Training set of OIELLM.
This Train set already formatted by OIELLM's format. The test set is in the another page in huggingface.
The MMM support 3 languages (English, Chinese and Japanese). And you must use task instruct words to define kind of task.
Mutual Reinforcement Effect.
OIELLM's input and output
MMM Dataset
The following is input and output format:
{
"input": "In 1953, filming of "On the Waterfront" starring… See the full description on the dataset page: https://huggingface.co/datasets/ganchengguang/MMM-datasets-Testset.SpatialGen-Testset
SpatialGen Testset
This repository contains the test set for SPATIALGEN: Layout-guided 3D Indoor Scene Generation, a novel multi-view multi-modal diffusion model for generating realistic and semantically consistent 3D indoor scenes.
Project page | Paper | Code
We provide a test set of 48 preprocessed point clouds and their corresponding GT layouts, multi-view images are cropped from the high-resolution panoramic images.
Folder Structure
Outlines of the dataset files:… See the full description on the dataset page: https://huggingface.co/datasets/manycore-research/SpatialGen-Testset.emu_edit_test_set_generations
Dataset Card for the Emu Edit Generations on Emu Edit Test Set
Dataset Summary
This dataset contains Emu Edit's generations on the Emu Edit test set. For more information please read our paper or visit our homepage.
Licensing Information
Licensed with CC-BY-NC 4.0 License available here.
Citation Information
@inproceedings{Sheynin2023EmuEP,
title={Emu Edit: Precise Image Editing via Recognition and Generation Tasks},
author={Shelly Sheynin and… See the full description on the dataset page: https://huggingface.co/datasets/facebook/emu_edit_test_set_generations.Tokenizer-Testsetexternal_test_set_v1TestSet1Mixture_0616testset_ellietestset_winogrande-mcqtest_import_dataset_from_hub_using_settings_with_recordsTrue
Dataset Card for test_import_dataset_from_hub_using_settings_with_recordsTrue
This dataset has been created with Argilla. As shown in the sections below, this dataset can be loaded into your Argilla server as explained in Load with Argilla, or used directly with the datasets library in Load with datasets.
Using this dataset with Argilla
To load with Argilla, you'll just need to install Argilla as pip install argilla --upgrade and then use the following code:
import… See the full description on the dataset page: https://huggingface.co/datasets/argilla-internal-testing/test_import_dataset_from_hub_using_settings_with_recordsTrue.testset_munchsmugri-flores-testsetMultilingual FLORES-based benchmark for Komi, Udmurt, Hill and Meadow Mari, Erzya, Moksha, Livonian, Mansi, and Livvi Karelian. Expanded with Proper Karelian, Ludian, and Veps.
Please, cite the following paper if you use Komi, Udmurt, Hill and Meadow Mari, Erzya, Moksha, Livonian, Mansi, and Livvi Karelian datasets:
@inproceedings{
yankovskaya2023machine,
title={Machine Translation for Low-resource Finno-Ugric Languages},
author={Lisa Yankovskaya and Maali Tars and Andre T{\"a}ttar and Mark… See the full description on the dataset page: https://huggingface.co/datasets/tartuNLP/smugri-flores-testset.test-data-set-Arabic-letterDr.Sparse-OTF-test-set
Dr.Sparse OTF Test Set
100 sparse matrices from the SuiteSparse Matrix Collection,
converted to the flat binary format the Dr.Sparse
benchmark harness reads. This is the held-out evaluation set for LLM-generated
CUDA sparse kernels (SpMV / SpMM / SpGEMM), kept separate from the matrices the
models were developed against.
Layout
Matrices are grouped into size tiers by row count, the convention Dr.Sparse task
discovery scans for:
tier
rows
matrices
size… See the full description on the dataset page: https://huggingface.co/datasets/KinGeorge/Dr.Sparse-OTF-test-set.winml-test-set
WinML Test Set
Dataset Summary
WinML Test Set is an evaluation‑only collection for validating model accuracy and stability on Windows ML / DirectML / ONNX Runtime pipelines. It aggregates several permissively‑licensed sources and harmonizes schema for reproducible, regression‑grade testing across backends and versions. Not intended for training.
Intended Use
Accuracy and regression benchmarking of Windows ML / DirectML / ONNX Runtime pipelines.… See the full description on the dataset page: https://huggingface.co/datasets/Futuremark/winml-test-set.vi-en-ast-testset