datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Multitask-National-Speech-Corpus-v1Multitask-National-Speech-Corpus (MNSC v1) is derived from IMDA's NSC Corpus.
MNSC is a multitask speech understanding dataset derived and further annotated from IMDA NSC Corpus. It focuses on the knowledge of Singapore's local accent, localised terms, and code-switching.
ASR: Automatic Speech Recognition
SQA: Speech Question Answering
SDS: Spoken Dialogue Summarization
PQA: Paralinguistic Question Answering
from datasets import load_dataset
data =… See the full description on the dataset page: https://huggingface.co/datasets/MERaLiON/Multitask-National-Speech-Corpus-v1.Multi-SWE-smith-tasksMultitask-National-Speech-Corpus-v1-extendmulti_task_multi_modal_knowledge_retrieval_benchmark_M2KR
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN.multi-task-tcc-robosuite
XIRL
Overview
Setup
Datasets
Code Navigation
Experiments: Reproducing Paper Results
Extending XIRL
Acknowledgments
Overview
Code release for our CoRL 2021 conference paper:
XIRL: Cross-embodiment Inverse Reinforcement Learning
Kevin Zakka1,3, Andy Zeng1, Pete Florence1, Jonathan Tompson1, Jeannette Bohg2, and Debidatta Dwibedi1
Conference on Robot Learning (CoRL) 2021
1Robotics at Google,
2Stanford… See the full description on the dataset page: https://huggingface.co/datasets/Renton-Ren/multi-task-tcc-robosuite.omx_gelsight_env1_multitask_horizontal_vertical_line_nogelThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "omx_follower",
"total_episodes": 120,
"total_frames": 105619,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:120"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/WoojongKim/omx_gelsight_env1_multitask_horizontal_vertical_line_nogel.eval_smolvla_policy_omx_gelsight_env1_multitask_horizontal_vertical_line_nogel_20260820_192704This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "omx_follower",
"total_episodes": 10,
"total_frames": 7847,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/WoojongKim/eval_smolvla_policy_omx_gelsight_env1_multitask_horizontal_vertical_line_nogel_20260820_192704.multitask_german_examples_32kp2-etf-multitask-gp-resultsso101_multi_taskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 9,
"total_frames": 3131,
"total_tasks": 1,
"total_videos": 18,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:9"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/fbeltrao/so101_multi_task.multitask_vqa_benchmarkThis dataset is a part of .
🍈 MMT-47: Multimodal Multi-Task Benchmark
47 Tasks · 7 Categories · 3 Modalities (Image, Video, Text)
Cite our ICML-2026 paper for this dataset
@article{kowsher2026lime,
title={LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning},
author={Kowsher, Md and Mansoor, Haris and Prottasha, Nusrat Jahan and Garibay, Ozlem and Zhu, Victor and Ji, Zhengping and Chen, Chen},
journal={arXiv preprint… See the full description on the dataset page: https://huggingface.co/datasets/Kowsher/multitask_vqa_benchmark.multitask-tabular-datasetsThis is a port of the Multi-Label Classification Dataset Repository (link).
We convert the datasets from there to simple csvs, resulting in 32 csvs (many of their mulan files fail to parse into python for us)
The targets in each csv are labeled with the suffix __target
Dataset
Domain
m
d
q
Card
Dens
Div
avgIR
rDep
m×q×d
0
3s-bbc1000
Text
352
1000
6
1.125
0.188
0.234
1.718
0.733
2.11e+06
1
3s-guardian1000
Text
302
1000
6
1.126
0.188
0.219
1.773
0.667
1.81e+06
2
3s-inter3000… See the full description on the dataset page: https://huggingface.co/datasets/imodels/multitask-tabular-datasets.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KRVQA-MultiTaskReal_World_Multi_Taskso101_multi_task_v2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 59,
"total_frames": 20844,
"total_tasks": 2,
"total_videos": 118,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:59"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/fbeltrao/so101_multi_task_v2.so101-multitask-calib
SO-101 multitask — calibration pool (LeRobot v2.1)
The observation pool that post-training quantization of a GR00T N1.7 SO-101
policy is calibrated on, in the LeRobot v2.1 layout. Same recordings as
hungho77/so101-multitask,
which is stored in the v3.0 layout; GR00T's data loader reads v2.1 only, so this
is the copy a quantization or evaluation run actually opens.
143 episodes · 67,496 frames · 3 tasks · 30 fps · single SO-101 arm ·
two 480×640 cameras (top, wrist) · 6-D state… See the full description on the dataset page: https://huggingface.co/datasets/hungho77/so101-multitask-calib.ur5e_multitask_000This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": null,
"total_episodes": 270,
"total_frames": 106900,
"total_tasks": 11,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:270"},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Perseus101/ur5e_multitask_000.so101_cube_multitask_hil_0724_merged_fixedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 175,
"total_frames": 209444,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:175"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Chaenn/so101_cube_multitask_hil_0724_merged_fixed.busybox_multitask
Single-arm BusyBox dataset
LeRobot v3 dataset containing 132 episodes and
25734 frames at 20 FPS across
27 BusyBox task(s), merged from 4 source
dataset(s). This revision adds 3 cell-3 source dataset(s) on top of armnet/busybox_multitask. Green-button cell-3 contributes only its first three episodes. Cell-3 language instructions are normalized to the original multitask strings.
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
The original unique… See the full description on the dataset page: https://huggingface.co/datasets/armnet/busybox_multitask.squad_multitask\Stanford Question Answering Dataset (SQuAD) is a reading comprehension \dataset, consisting of questions posed by crowdworkers on a set of Wikipedia \articles, where the answer to every question is a segment of text, or span, \from the corresponding reading passage, or the question might be unanswerable.franka_multi_task_v1_vidThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": null,
"total_episodes": 200,
"total_frames": 75843,
"total_tasks": 4,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Beegbrain/franka_multi_task_v1_vid.MultiTaskVideoReasoning
Multi Task Video Reasoning Dataset
This is the official training dataset for Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning.
[Project]
[arXiv]
[Code]
Data Structure
└── MultiTaskVideoReasoning
├── MTVR_CoT
│ ├── actnet.json
│ ├── charades.json
│ ├── longvideo-reason.json
│ ├── nextgqa.json
│ ├── rextime.json
│ ├── vidchapters.json
│ ├── Video-R1-data-image.json
│ └──… See the full description on the dataset page: https://huggingface.co/datasets/zhang9302002/MultiTaskVideoReasoning.franka_multi_task_v1_imgThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": null,
"total_episodes": 200,
"total_frames": 75843,
"total_tasks": 4,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Beegbrain/franka_multi_task_v1_img.multitask_5blocks_v2_530epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v2_530ep.ur10e-multitask
UR10e Multitask Teleop
335 teleoperated manipulation episodes on a Universal Robots UR10e with a Robotiq 2F gripper, covering 7 tabletop tasks. Packaged in LeRobot v2.1 format with precomputed normalization statistics, so it can be dropped into a VLA finetuning run the same way LIBERO is.
Episodes
335
Frames
60,416
Tasks
7
Control rate
15 fps
Robot
UR10e, 6-DoF, Robotiq 2F gripper
Cameras
2 exterior (Azure Kinect) + 1 wrist (RealSense)
Image size
180 x… See the full description on the dataset page: https://huggingface.co/datasets/bag100/ur10e-multitask.busybox_bimanual_multitask
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so-101",
"total_episodes": 56,
"total_frames": 13030,
"total_tasks": 24,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 20,
"splits": {
"train": "0:56"
},
"data_path":… See the full description on the dataset page: https://huggingface.co/datasets/villekuosmanen/busybox_bimanual_multitask.so101-multi-taskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/khanhnd61/so101-multi-task.logu_bimanual_joint_multi_taskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": null,
"total_episodes": 212,
"total_frames": 101703,
"total_tasks": 2,
"total_videos": 636,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 20,
"splits": {
"train": "0:212"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/gaozj/logu_bimanual_joint_multi_task.