datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Video-IFBench
Video-IFBench
This release contains the evaluation split used for the Video-IFBench main experiments.
Paper: Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Project page: https://alexios-hub.github.io/Video-IFBench/
Code: https://github.com/Alexios-hub/Video-IFBench
OmniCap-IF
OmniCap-IF
OmniCap-IF is a benchmark for evaluating instruction following in omni-modal video captioning. It contains 480 videos and 1,920 instruction samples spanning understanding, generation, retrieval, and communication-oriented captioning tasks. Each sample pairs a prompt with fine-grained format and content checklists for evaluating whether a model follows structural, temporal, visual, audio, and audio-visual constraints.… See the full description on the dataset page: https://huggingface.co/datasets/NJU-LINK/OmniCap-IF.IF-VidCap
IF-VidCap:
Can Video Caption Models Follow Instructions?
English | 中文
📋 Abstract
Although Multimodal Large Language Models (MLLMs) have demonstrated proficiency in video captioning, practical applications require captions that follow specific user instructions rather than generating exhaustive, unconstrained descriptions. Current benchmarks, however, primarily assess descriptive comprehensiveness while largely overlook… See the full description on the dataset page: https://huggingface.co/datasets/NJU-LINK/IF-VidCap.gazebo_push_box_3dof_v1my_data_newThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "fr3",
"total_episodes": 5,
"total_frames": 1607,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 15,
"splits": {
"train": "0:5"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iftekher/my_data_new.my_data_handThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "fr3",
"total_episodes": 5,
"total_frames": 1227,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 15,
"splits": {
"train": "0:5"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iftekher/my_data_hand.gazebo_push_box_3dof_v2so101_IF_2
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Move the [object name] [left/right/forwards/backwards]"
gazebo_push_t_3dof_v1gazebo_lerobot_push_box_3dof_v2_96_10epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 10,
"total_frames": 3875,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v2_96_10ep.pick_and_place_ifa_noopThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Dangni/pick_and_place_ifa_noop.so101_IF_1
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Move the [object name] [left/right/forwards/backwards]"
gazebo_lerobot_push_box_3dof_v2_512_10epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 10,
"total_frames": 3875,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:10"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v2_512_10ep.gazebo_lerobot_push_box_3dof_v2_96_101epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 101,
"total_frames": 31844,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:101"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v2_96_101ep.pick_and_place_ifa2so101_IF_6
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Place the [object 1] between the [object 2] and the [object 3]"
"Move the [object name] [left/right/forwards/backwards]"
25 household objects, with each episode's task controlled for ambiguity (e.g preventing "Place the pen in the plastic bin" with two pens on the table)
pick_and_place_ifagazebo_lerobot_push_t_3dof_v1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 21,
"total_frames": 24930,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:21"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_t_3dof_v1.gazebo_lerobot_push_box_3dof_v1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 31,
"total_frames": 13725,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:31"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v1.so101_IF_7
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Place the [object 1] between the [object 2] and the [object 3]"
"Move the [object name] [left/right/forwards/backwards]"
25 household objects, with each episode's task controlled for ambiguity (e.g preventing "Place the pen in the plastic bin" with two pens on the table)
gazebo_lerobot_push_box_3dof_v2_100epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 101,
"total_frames": 41084,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:101"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v2_100ep.gazebo_lerobot_push_box_3dof_v2_512_101epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "3dof_planar_arm",
"total_episodes": 101,
"total_frames": 31844,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 10,
"splits": {
"train": "0:101"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/iFaz/gazebo_lerobot_push_box_3dof_v2_512_101ep.149-TUHH-IFPT-lerobot-shadow-boardThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 58,
"total_frames": 75259,
"total_tasks": 1,
"total_videos": 174,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:58"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/LeRobot-worldwide-hackathon/149-TUHH-IFPT-lerobot-shadow-board.so101_IF_3
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Move the [object name] [left/right/forwards/backwards]"
so101_IF_5
Tasks:
"Place the [object name] in the plastic bin"
"Place the [object 1] next to the [object 2]"
"Place the [object 1] between the [object 2] and the [object 3]"
"Move the [object name] [left/right/forwards/backwards]"
25 household objects, with each episode's task controlled for ambiguity (e.g preventing "Place the pen in the plastic bin" with two pens on the table)
pupupu_iffice4090This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_so101_follower",
"total_episodes": 1,
"total_frames": 312,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/msmandelbrot/pupupu_iffice4090.pick_and_stack_20260520_141603This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ifuxxloveu/pick_and_stack_20260520_141603.so101_IF_2_v2.1pick_and_place_20260518_105002This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ifuxxloveu/pick_and_place_20260518_105002.pick_and_place_20260518_111934This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ifuxxloveu/pick_and_place_20260518_111934.