datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Multitask-National-Speech-Corpus-v1Multitask-National-Speech-Corpus (MNSC v1) is derived from IMDA's NSC Corpus.
MNSC is a multitask speech understanding dataset derived and further annotated from IMDA NSC Corpus. It focuses on the knowledge of Singapore's local accent, localised terms, and code-switching.
ASR: Automatic Speech Recognition
SQA: Speech Question Answering
SDS: Spoken Dialogue Summarization
PQA: Paralinguistic Question Answering
from datasets import load_dataset
data =… See the full description on the dataset page: https://huggingface.co/datasets/MERaLiON/Multitask-National-Speech-Corpus-v1.Multitask-National-Speech-Corpus-v1-extendmulti_task_multi_modal_knowledge_retrieval_benchmark_M2KR
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN.multi-task-tcc-robosuite
XIRL
Overview
Setup
Datasets
Code Navigation
Experiments: Reproducing Paper Results
Extending XIRL
Acknowledgments
Overview
Code release for our CoRL 2021 conference paper:
XIRL: Cross-embodiment Inverse Reinforcement Learning
Kevin Zakka1,3, Andy Zeng1, Pete Florence1, Jonathan Tompson1, Jeannette Bohg2, and Debidatta Dwibedi1
Conference on Robot Learning (CoRL) 2021
1Robotics at Google,
2Stanford… See the full description on the dataset page: https://huggingface.co/datasets/Renton-Ren/multi-task-tcc-robosuite.omx_gelsight_env1_multitask_horizontal_vertical_line_nogelThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "omx_follower",
"total_episodes": 120,
"total_frames": 105619,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:120"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/WoojongKim/omx_gelsight_env1_multitask_horizontal_vertical_line_nogel.eval_smolvla_policy_omx_gelsight_env1_multitask_horizontal_vertical_line_nogel_20260820_192704This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "omx_follower",
"total_episodes": 10,
"total_frames": 7847,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/WoojongKim/eval_smolvla_policy_omx_gelsight_env1_multitask_horizontal_vertical_line_nogel_20260820_192704.multitask_german_examples_32kp2-etf-multitask-gp-resultsmultitask_vqa_benchmarkThis dataset is a part of .
🍈 MMT-47: Multimodal Multi-Task Benchmark
47 Tasks · 7 Categories · 3 Modalities (Image, Video, Text)
Cite our ICML-2026 paper for this dataset
@article{kowsher2026lime,
title={LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning},
author={Kowsher, Md and Mansoor, Haris and Prottasha, Nusrat Jahan and Garibay, Ozlem and Zhu, Victor and Ji, Zhengping and Chen, Chen},
journal={arXiv preprint… See the full description on the dataset page: https://huggingface.co/datasets/Kowsher/multitask_vqa_benchmark.multitask-tabular-datasetsThis is a port of the Multi-Label Classification Dataset Repository (link).
We convert the datasets from there to simple csvs, resulting in 32 csvs (many of their mulan files fail to parse into python for us)
The targets in each csv are labeled with the suffix __target
Dataset
Domain
m
d
q
Card
Dens
Div
avgIR
rDep
m×q×d
0
3s-bbc1000
Text
352
1000
6
1.125
0.188
0.234
1.718
0.733
2.11e+06
1
3s-guardian1000
Text
302
1000
6
1.126
0.188
0.219
1.773
0.667
1.81e+06
2
3s-inter3000… See the full description on the dataset page: https://huggingface.co/datasets/imodels/multitask-tabular-datasets.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KRVQA-MultiTaskso101-multitask-calib
SO-101 multitask — calibration pool (LeRobot v2.1)
The observation pool that post-training quantization of a GR00T N1.7 SO-101
policy is calibrated on, in the LeRobot v2.1 layout. Same recordings as
hungho77/so101-multitask,
which is stored in the v3.0 layout; GR00T's data loader reads v2.1 only, so this
is the copy a quantization or evaluation run actually opens.
143 episodes · 67,496 frames · 3 tasks · 30 fps · single SO-101 arm ·
two 480×640 cameras (top, wrist) · 6-D state… See the full description on the dataset page: https://huggingface.co/datasets/hungho77/so101-multitask-calib.ur5e_multitask_000This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": null,
"total_episodes": 270,
"total_frames": 106900,
"total_tasks": 11,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:270"},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Perseus101/ur5e_multitask_000.so101_cube_multitask_hil_0724_merged_fixedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 175,
"total_frames": 209444,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:175"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Chaenn/so101_cube_multitask_hil_0724_merged_fixed.busybox_multitask
Single-arm BusyBox dataset
LeRobot v3 dataset containing 132 episodes and
25734 frames at 20 FPS across
27 BusyBox task(s), merged from 4 source
dataset(s). This revision adds 3 cell-3 source dataset(s) on top of armnet/busybox_multitask. Green-button cell-3 contributes only its first three episodes. Cell-3 language instructions are normalized to the original multitask strings.
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
The original unique… See the full description on the dataset page: https://huggingface.co/datasets/armnet/busybox_multitask.squad_multitask\Stanford Question Answering Dataset (SQuAD) is a reading comprehension \dataset, consisting of questions posed by crowdworkers on a set of Wikipedia \articles, where the answer to every question is a segment of text, or span, \from the corresponding reading passage, or the question might be unanswerable.MultiTaskVideoReasoning
Multi Task Video Reasoning Dataset
This is the official training dataset for Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning.
[Project]
[arXiv]
[Code]
Data Structure
└── MultiTaskVideoReasoning
├── MTVR_CoT
│ ├── actnet.json
│ ├── charades.json
│ ├── longvideo-reason.json
│ ├── nextgqa.json
│ ├── rextime.json
│ ├── vidchapters.json
│ ├── Video-R1-data-image.json
│ └──… See the full description on the dataset page: https://huggingface.co/datasets/zhang9302002/MultiTaskVideoReasoning.multitask_5blocks_v2_530epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v2_530ep.ur10e-multitask
UR10e Multitask Teleop
335 teleoperated manipulation episodes on a Universal Robots UR10e with a Robotiq 2F gripper, covering 7 tabletop tasks. Packaged in LeRobot v2.1 format with precomputed normalization statistics, so it can be dropped into a VLA finetuning run the same way LIBERO is.
Episodes
335
Frames
60,416
Tasks
7
Control rate
15 fps
Robot
UR10e, 6-DoF, Robotiq 2F gripper
Cameras
2 exterior (Azure Kinect) + 1 wrist (RealSense)
Image size
180 x… See the full description on the dataset page: https://huggingface.co/datasets/bag100/ur10e-multitask.busybox_bimanual_multitask
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so-101",
"total_episodes": 56,
"total_frames": 13030,
"total_tasks": 24,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 20,
"splits": {
"train": "0:56"
},
"data_path":… See the full description on the dataset page: https://huggingface.co/datasets/villekuosmanen/busybox_bimanual_multitask.xlerobot_multitask_part13This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": null,
"total_episodes": 21,
"total_frames": 13987,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 0.001,
"fps": 30,
"splits": {
"train": "0:21"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/ArthurWangSawau/xlerobot_multitask_part13.smolvla_multitask_hil_285k_v1_220ep_trimmedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/smolvla_multitask_hil_285k_v1_220ep_trimmed.mission2_smolvla_multitask_v2_120epcoastal-multitask-380
Coastal & Rural Bangladesh — Multi-Task Visual Dataset
379 field photographs (JPEG, native resolution as shot — see classification/metadata.csv for per-image width/height) collected on foot along the Bakkhali river embankment and surrounding villages/farmland near Cox's Bazar, Bangladesh, structured into three ML-task "levels": classification, semantic segmentation, and change detection.
Source: huggingface data 06 (Golam Rob / Tawhid Enterprise photo collection).… See the full description on the dataset page: https://huggingface.co/datasets/golamrob/coastal-multitask-380.GPRadar-Defect-MultiTask
GPRadar-Defect-MultiTask 数据集
本仓库包含用于微调PaLI-GEMMA多模态模型的地质雷达(GPR)缺陷检测数据集。该数据集专注于地下结构中的空洞和裂缝检测与分析。
数据集结构
数据集组织如下：
dataset/
├── annotations/ - 包含JSON和JSONL格式的标注文件
│ ├── _annotations.train.jsonl - 训练集标注
│ ├── _annotations.valid.jsonl - 验证集标注
│ ├── _annotations.test.jsonl - 测试集标注
│ ├── p-1.v1i.paligemma/ - 主数据集元数据
│ └── p-1.v1i.paligemma-multimodal/ - 多模态数据集元数据
├── images/ - 包含所有图像文件
特点
包含874张带注释的地质雷达扫描图像
图像预处理为640x640像素大小
支持多种任务类型：缺陷检测、位置定位和描述生成… See the full description on the dataset page: https://huggingface.co/datasets/xingqiang/GPRadar-Defect-MultiTask.paligemma-multitask-dataset
PaliGemma Multitask Dataset
This dataset is designed for training and evaluating the PaliGemma multitask model for defect detection and analysis. It combines a base set of annotated samples with an extended collection of 874 real-world structural inspection images.
Dataset Description
Overview
The dataset contains images of structural defects along with their corresponding annotations for:
Object detection (bounding boxes)
Defect classification… See the full description on the dataset page: https://huggingface.co/datasets/xingqiang/paligemma-multitask-dataset.multitask_5blocks_v1_444epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v1_444ep.Sampling-Multitask-National-Speech-Corpus-v1
Sampling Multitask-National-Speech-Corpus-v1
Original dataset from https://huggingface.co/datasets/MERaLiON/Multitask-National-Speech-Corpus-v1, we only take Part 3 and do sampling.
how to prepare the dataset
huggingface-cli download \
mesolitica/Sampling-Multitask-National-Speech-Corpus-v1 \
--include "*.zip" \
--repo-type "dataset" \
--local-dir './'
wget… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Sampling-Multitask-National-Speech-Corpus-v1.