datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN.NEXUS-temporal_hierarchical_multi-modal
NEXUS: Neural Evolution for eXtensible Universal Semantics Dataset
(Temporal Multimodal Slices)
This dataset is a multi-modal, hierarchical, temporal representation derived from HuggingFaceFV/finevideo. It is designed for streaming training where the primary unit is a 10 ms "slice" that aggregates upward into moments (100 ms), seconds (1 s), experiences (10 s), and minutes (60 s).
It is meant to represent an extensible stream of "experience" as there are… See the full description on the dataset page: https://huggingface.co/datasets/Ardea/NEXUS-temporal_hierarchical_multi-modal.SAE_activations_modal_sentencesbimanual-table-cleanup-cross-embodiment-rich-modality-sample
Cross-Embodiment Bimanual Table Cleanup — Rich-Modality 10-Episode Inspection Sample
10 full-modality cross-embodiment bimanual table-cleanup episodes: 5 Franka Panda + 5 WidowXAI, 21,267 frames, 6 RGB views per robot, task-camera depth and segmentation, native robot state/action, end-effector trajectories, 6-DoF object poses, and QA annotations.
✅ Use it / ❌ Skip it
Use it for
Inspecting loaders, schemas, camera coverage, depth, segmentation, object poses… See the full description on the dataset page: https://huggingface.co/datasets/ExylosAi/bimanual-table-cleanup-cross-embodiment-rich-modality-sample.multi-modal-derived-brain-network
PPMI Connectivity Graphs — HF Staging (Derivatives)
This dataset ships ready-to-use functional brain connectivity graphs derived from the PPMI cohort in a BIDS-ish derivatives layout. For each subject and parcellation, we include:
ROI time-series (*_desc-timeseries_parc-<name>.mat)
Pearson correlation connectivity matrix (*_desc-correlation_matrix_parc-<name>.mat)
JSON sidecars with summary fields (nodes, measure, symmetric/weighted flags)
Contents
data/… See the full description on the dataset page: https://huggingface.co/datasets/pakkinlau/multi-modal-derived-brain-network.object-sorting-cross-embodiment-rich-modality-sample
Cross-Embodiment Object Sorting — Rich-Modality 20-Episode Inspection Sample
20 episodes total: 10 Franka Panda + 10 WidowXAI. A compact cross-embodiment inspection release for picking up an instructed object and placing it into an instructed target box. Each robot keeps its native LeRobot v2.1 state/action schema in a separate Viewer config. Five synchronized RGB views, metric depth and instance segmentation for every camera, robot state/action, end-effector trajectories… See the full description on the dataset page: https://huggingface.co/datasets/ExylosAi/object-sorting-cross-embodiment-rich-modality-sample.coffee-brew-cross-embodiment-rich-modality-sample
Cross-Embodiment Coffee Brew — Rich-Modality 10-Episode Inspection Sample
10 full-modality cross-embodiment coffee-brew episodes: 5 single-arm Franka Panda + 5 single-arm WidowXAI, 14,401 frames, 5 RGB views per robot, task-camera depth and segmentation, native robot state/action, end-effector trajectories, 6-DoF object poses, and QA annotations.
✅ Use it / ❌ Skip it
Use it for
Inspecting loaders, schemas, camera coverage, depth, segmentation, object poses… See the full description on the dataset page: https://huggingface.co/datasets/ExylosAi/coffee-brew-cross-embodiment-rich-modality-sample.multi-modal-peg-in-square-hole-testThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "ur5",
"total_episodes": 51,
"total_frames": 8074,
"total_tasks": 1,
"total_videos": 153,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:51"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hainh22/multi-modal-peg-in-square-hole-test.multi-modal-peg-in-square-hole-image-aloneThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "ur5",
"total_episodes": 11,
"total_frames": 1892,
"total_tasks": 1,
"total_videos": 33,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:11"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hainh22/multi-modal-peg-in-square-hole-image-alone.full-modality-data
Full Modality Dataset Statistics
Video Statistics
Total Videos: 28,472
Total Duration: 1422.33 hours
Average Duration: 179.84 seconds
Median Duration: 160.08 seconds
Duration Range: 10.04s - 1780.03s
QA Statistics
Total Questions: 1,444,526
Average Questions per Video: 50.7
Questions per Video Range: 14 - 450
Question Type Distribution
OE: 1,444,526 (100.0%)
Question Category Distribution
temporal: 96,873 (6.7%)
causal: 96,873… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab/full-modality-data.mod-alg2
Dataset Card for "mod-alg2"
More Information needed
rollout_act_so101_earplug_cup_v3_modal_20260702_155144This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/AlexTranscend/rollout_act_so101_earplug_cup_v3_modal_20260702_155144.multi_modality_testThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "koch",
"total_episodes": 6,
"total_frames": 2543,
"total_tasks": 1,
"total_videos": 18,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:6"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/seeingrain/multi_modality_test.multi-modal-peg-in-square-holeThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "ur5",
"total_episodes": 201,
"total_frames": 28616,
"total_tasks": 1,
"total_videos": 603,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:201"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hainh22/multi-modal-peg-in-square-hole.so101_teleop_mat_to_tray_with_modalitiesThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 10,
"total_frames": 2020,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/liorbenhorin-nv/so101_teleop_mat_to_tray_with_modalities.rollout_act_so101_earplug_cup_v3_modal_20260702_155430This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/AlexTranscend/rollout_act_so101_earplug_cup_v3_modal_20260702_155430.rollout_act_so101_earplug_cup_v3_modal_20260702_155854This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/AlexTranscend/rollout_act_so101_earplug_cup_v3_modal_20260702_155854.europe-owid-modal-age-at-death-in-adults
Modal Age At Death In Adults | Europe (Our World in Data)
🇪🇺 238 observations · 5 Europe countries · 1970–2022 · Repackaged by Electric Sheep Europe
TL;DR
This dataset contains 238 observations of Modal Age At Death In Adults data across 5 Europe countries, spanning 1970–2022.
About the source
Source: Our World in Data
Publisher: Our World in Data
License: cc-by-4.0
Topic: Modal Age At Death In Adults
Geographic coverage
5… See the full description on the dataset page: https://huggingface.co/datasets/electricsheepeurope/europe-owid-modal-age-at-death-in-adults.MULTI_VALUE_qqp_present_modals
Dataset Card for "MULTI_VALUE_qqp_present_modals"
More Information needed
new_screw_ticks_yolo_modalityThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "mcx",
"total_episodes": 2,
"total_frames": 1748,
"total_tasks": 1,
"total_videos": 4,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/mortiferr/new_screw_ticks_yolo_modality.MULTI_VALUE_mrpc_present_modals
Dataset Card for "MULTI_VALUE_mrpc_present_modals"
More Information needed
MULTI_VALUE_cola_double_modals
Dataset Card for "MULTI_VALUE_cola_double_modals"
More Information needed
MULTI_VALUE_mrpc_double_modals
Dataset Card for "MULTI_VALUE_mrpc_double_modals"
More Information needed
MULTI_VALUE_mnli_present_modals
Dataset Card for "MULTI_VALUE_mnli_present_modals"
More Information needed
MULTI_VALUE_sst2_double_modals
Dataset Card for "MULTI_VALUE_sst2_double_modals"
More Information needed
MULTI_VALUE_mnli_double_modals
Dataset Card for "MULTI_VALUE_mnli_double_modals"
More Information needed
MULTI_VALUE_cola_present_modals
Dataset Card for "MULTI_VALUE_cola_present_modals"
More Information needed
new_screw_ticks_yolo_modalityThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "mcx",
"total_episodes": 2,
"total_frames": 1748,
"total_tasks": 1,
"total_videos": 4,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/antwoor/new_screw_ticks_yolo_modality.MULTI_VALUE_wnli_present_modals
Dataset Card for "MULTI_VALUE_wnli_present_modals"
More Information needed