datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR.prof_report__wavymulder-Analog-Diffusion__multi__24
Dataset Card for "prof_report__wavymulder-Analog-Diffusion__multi__24"
More Information needed
Multi-SWE-RL-Verified
Multi-SWE-RL-Verified
Gold-patch-validated subset of
PrimeIntellect/Multi-SWE-RL-Reupload
(ByteDance's Multi-SWE-RL): 2,232 / 4,703 rows across
C, Go, Java, JavaScript, Rust, and TypeScript that produce a clean reward signal end-to-end.
Default dataset of the multiswe_v1 taskset.
Changes vs upstream
Starting from the 4,703-row re-upload:
C++ dropped wholesale — 0/449 rows passed gold-patch validation in pass 1; the images are
broken for scoring, not merely… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Multi-SWE-RL-Verified.plasticc---
description: 'The Photometric LSST Astronomical Time-Series Classification Challenge
(PLAsTiCC) is a community-wide challenge to spur development of algorithms to classify
astronomical transients. The Large Synoptic Survey Telescope (LSST) will discover
tens of thousands of transient phenomena every single night. To deal with this massive
onset of data, automated algorithms to classify and sort astronomical transients
are crucial.
'
homepage: https://zenodo.org/records/2539456… See the full description on the dataset page: https://huggingface.co/datasets/MultimodalUniverse/plasticc.prof_report__22h-vintedois-diffusion-v0-1__multi__24
Dataset Card for "prof_report__22h-vintedois-diffusion-v0-1__multi__24"
More Information needed
MultiPL-E-completions
Raw Data from MultiPL-E
This repository is frozen. See https://huggingface.co/datasets/nuprl/MultiPL-E-completions for a more complete version of this repository.
Uploads are a work in progress. If you are interested in a split that is not yet available, please contact a.guha@northeastern.edu.
This repository contains the raw data -- both completions and executions -- from MultiPL-E that was used to generate several experimental results from the
MultiPL-E, SantaCoder, and StarCoder… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/MultiPL-E-completions.ledger-long-context-multi-kpi
the LEDGER Long-Context Multi-KPI extraction datasets and benchmarks.
OCR'd annual reports with ground-truth KPI values for financial information extraction benchmarking.
Dataset Description
This dataset pairs OCR-extracted annual report text (from DeepSeek OCR) with structured KPI ground-truth values. It is designed for evaluating LLM-based financial information extraction, retrieval, and needle-in-a-haystack tasks.
Configs
Config
Reports… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-multi-kpi.synthetic_pii_finance_multilingual
Image generated by DALL-E. See prompt for more details
💼 📊 Synthetic Financial Domain Documents with PII Labels
gretelai/synthetic_pii_finance_multilingual is a dataset of full length synthetic financial documents containing Personally Identifiable Information (PII), generated using Gretel Navigator and released under Apache 2.0.
This dataset is designed to assist with the following use cases:
🏷️ Training NER (Named Entity Recognition) models to detect and label PII in… See the full description on the dataset page: https://huggingface.co/datasets/gretelai/synthetic_pii_finance_multilingual.stanford_kuka_multimodal_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 3000,
"total_frames": 149985,
"total_tasks": 1,
"total_videos": 3000,
"total_chunks": 3,
"chunks_size": 1000,
"fps": 20,
"splits": {
"train": "0:3000"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/stanford_kuka_multimodal_dataset.MultiTurn-Chat-MT-Bench-Judge
SEA-MT-Bench-Judge
SEA-MT-Bench-Judge expands on the original SEA-MTBench through the use of a criteria-based evaluation framework. We use GPT-OSS-120B as the judge model.
The prompts are based on MT-Bench and was manually translated by native speakers. Furthermore, some prompts were modified to be more suitable for the criteria-based judgments.
Supported Tasks and Leaderboards
SEA-MT-Bench-Judge is designed for evaluating chat or instruction-tuned large language… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/MultiTurn-Chat-MT-Bench-Judge.multi_session_chat
Dataset Card for "multi_session_chat"
More Information needed
rag_multilingual_training_negatives
How this dataset was made
We trained on chunks sourced from the documents in MADLAD-400 dataset that had been evaluated to contain a higher amount of educational information according to a state-of-the-art LLM.
We took chunks of size 250 tokens, 500 tokens, and 1000 tokens randomly for each document.
We then used these chunks to generate questions and answers based on this text using a state-of-the-art LLM.
Finally, we selected negatives for each chunk using the similarity from the… See the full description on the dataset page: https://huggingface.co/datasets/lightblue/rag_multilingual_training_negatives.MMLU_multi_promptrelaion2B-multi-research-safeEgoDex-PickPlace-YAM-14dof-multiview
EgoDex → YAM 14-DOF, Multiview (LeRobot v2.1)
Egocentric human hand-manipulation demonstrations from EgoDex retargeted to a
YAM bimanual robot (14-DOF), packaged as a LeRobot v2.1 dataset with three
synthesized camera views. The observation schema is drop-in compatible with
angkul07/abc-teleop for
cotraining (identical Hz, camera keys, and action convention).
At a glance
Episodes
8,842
Frames
1,074,893
Control rate
30 Hz (30 fps video)
Robot
YAM… See the full description on the dataset page: https://huggingface.co/datasets/angkul07/EgoDex-PickPlace-YAM-14dof-multiview.prof_report__CompVis-stable-diffusion-v1-4__multi__24
Dataset Card for "prof_report__CompVis-stable-diffusion-v1-4__multi__24"
More Information needed
multi-view-bathroom-scene-understanding-camera-relocalization
Multi-View Bathroom Scene Understanding & Camera Relocalization
Generated by datapack-import.ts
This dataset mirrors public data-pack render outputs from Physicl.
Each row represents one render view. The image column contains a stable URL to the primary render image uploaded under /data; image_path stores the relative repository path and data_commit_sha pins the Hugging Face dataset commit used by those URLs. Files are uploaded as downloaded unless optional PNG recompression is… See the full description on the dataset page: https://huggingface.co/datasets/physicl/multi-view-bathroom-scene-understanding-camera-relocalization.prof_report__plasmo-vox2__multi__24
Dataset Card for "prof_report__plasmo-vox2__multi__24"
More Information needed
MultiPL-E-completions
Raw Data from MultiPL-E
This repository contains the raw data -- both completions and executions --
from MultiPL-E that was used to generate several experimental results from the
MultiPL-E, SantaCoder, and StarCoder papers.
The original MultiPL-E completions and executions are stored in JOSN files. We use the following script
to turn each experiment directory into a dataset split and upload to this repository.
Every split is named base_dataset.language.model.temperature.variation… See the full description on the dataset page: https://huggingface.co/datasets/nuprl/MultiPL-E-completions.wp-multiteacher-distill-v1multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN.prof_report__andite-pastel-mix__multi__24
Dataset Card for "prof_report__andite-pastel-mix__multi__24"
More Information needed
multilingual-reward-bench
Multilingual Reward Bench (v1.0)
Reward models (RMs) have driven the development of state-of-the-art LLMs today, with unprecedented impact across the globe. However, their performance in multilingual settings still remains understudied.
In order to probe reward model behavior on multilingual data, we present M-RewardBench, a benchmark for 23 typologically diverse languages.
M-RewardBench contains prompt-chosen-rejected preference triples obtained by curating and translating chat… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabsCommunity/multilingual-reward-bench.relaion2B-multi-researchreal-routing-d2-r00-r05-eval
Route Cable: round 0-5 evaluation
Real-robot held-out evaluation for the Mulligan paper, one dataset per task-round.
Episodes from every evaluation session of this round are joined on the initial state
(meta/start_table.parquet: one row per start, one column group per policy).
Sessions
session_id is the fixed release block ID of the recording; it never gets renumbered.
Session
Results saved
Sobol seed
Start indices
Cameras
Original recording
b01… See the full description on the dataset page: https://huggingface.co/datasets/mulligan/real-routing-d2-r00-r05-eval.prof_report__SG161222-Realistic_Vision_V1.4__multi__24
Dataset Card for "prof_report__SG161222-Realistic_Vision_V1.4__multi__24"
More Information needed
NEXUS-temporal_hierarchical_multi-modal
NEXUS: Neural Evolution for eXtensible Universal Semantics Dataset
(Temporal Multimodal Slices)
This dataset is a multi-modal, hierarchical, temporal representation derived from HuggingFaceFV/finevideo. It is designed for streaming training where the primary unit is a 10 ms "slice" that aggregates upward into moments (100 ms), seconds (1 s), experiences (10 s), and minutes (60 s).
It is meant to represent an extensible stream of "experience" as there are… See the full description on the dataset page: https://huggingface.co/datasets/Ardea/NEXUS-temporal_hierarchical_multi-modal.prof_report__Lykon-DreamShaper__multi__24
Dataset Card for "prof_report__Lykon-DreamShaper__multi__24"
More Information needed
egocentric-vr-capture-20h-multimodal-sample
Egocentric VR Capture — 20-Hour Multimodal Inspection Sample
195 real-world task episodes / 2,283,482 frames / 21.14 delivered hours captured with consumer VR hardware. Each episode combines egocentric RGB and audio with synchronized headset, camera, body, and hand tracking in a LeRobot v3-style package.
This publicly accessible 20-hour-scale dataset is produced by the EXYLOS real-world data pipeline. Files and the Dataset Viewer can be accessed without individual approval;… See the full description on the dataset page: https://huggingface.co/datasets/ExylosAi/egocentric-vr-capture-20h-multimodal-sample.smolvla_multiblockThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "koch_follower",
"total_episodes": 10,
"total_frames": 4437,
"total_tasks": 1,
"total_videos": 20,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/ethanCSL/smolvla_multiblock.