datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
RMBench-taco-gemini
RMBench-taco-gemini
RMBench training episodes (9 tasks, 449 episodes, 30 fps) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 25 frames) and labels every sampled frame under the
task-specific context (taco) induced for that task. Each tick carries the current subtask, the running textual memory and the
visual-memory operations (keyframe store / retrieval) that the online… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RMBench-taco-gemini.RMBench-preset-gemini
RMBench-preset-gemini
RMBench training episodes (9 tasks, 450 episodes, 30 fps) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 25 frames) and labels every sampled frame given only
the task's subtask preset (the ordered list of subtask labels, no further task-specific guidance). Each tick carries the current
subtask, the running textual memory and the visual-memory operations… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RMBench-preset-gemini.gemma-4-e4b-webvid-4K
gemma-4-e4b-webvid-4K
This dataset contains the webvid_upgraded.json annotations and the videos referenced by that file.
Source: https://huggingface.co/datasets/OpenGVLab/VideoChat2-IT/tree/main/video/vqa/webvid_qa.
Files
webvid_upgraded.json: upgraded WebVid QA/action annotations.
videos/: MP4 files referenced by webvid_upgraded.json.
All video paths in webvid_upgraded.json are relative to the dataset root and point into videos/, for example… See the full description on the dataset page: https://huggingface.co/datasets/bear7011/gemma-4-e4b-webvid-4K.RoboDojo-taco-visual-gemini
RoboDojo-taco-visual-gemini
The visual-grounding variant of RoboDojo-taco-gemini: the same
RoboDojo long-horizon episodes (8 tasks, 800 episodes, 25 fps) with the same dense high-level labels (Gemini 3.7 Flash under the
task-specific context induced for each task), except that a target position leaves the label text and is drawn into the
low-level policy's keyframe slot.
In the source labels a target that words cannot identify is named by its image coordinates on the 0–1000… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RoboDojo-taco-visual-gemini.RMBench-taco-wodemo-gemini
RMBench-taco-wodemo-gemini
RMBench training episodes (9 tasks, 450 episodes, 30 fps) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 25 frames) and labels every sampled frame under a
task-specific context (taco) for that task. Each tick carries the current subtask, the running textual memory and the
visual-memory operations (keyframe store / retrieval) that the online high-level… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RMBench-taco-wodemo-gemini.thanos_picking_power_gemThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 51,
"total_frames": 16267,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:51"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/imstevenpmwork/thanos_picking_power_gem.GEM_gaze-assisted-ego-motion-in-drivingcs2-v3-prompt-comparison-7-examples-with-multiaction-gemini35
CS2 V3 七案例 Gemini 3.5 Flash 最终结果对比 / Seven-case Gemini 3.5 Flash Comparison
本 README 展示 Gemini 3.5 Flash 对同一批 7 个视频的最终打标结果：每个案例先显示视频，再用左右两列并排展示两轮和七轮的完整 English JSON 与中文 JSON；内容直接展开，字号保持较小以便对照。
This README shows Gemini 3.5 Flash final labels for the same 7 videos. Each case places the video first, then displays complete English and Chinese JSON side by side: two-round on the left and seven-round on the right.
两轮与七轮的 API 输入详情通过顶部索引查看；中文侧保持与英文 JSON 相同的键、时间边界、数组长度和 Action 标签。
API… See the full description on the dataset page: https://huggingface.co/datasets/mikusama99/cs2-v3-prompt-comparison-7-examples-with-multiaction-gemini35.gemma_rover_scoop_up_to_5
gemma_rover_scoop_up_to_5
This dataset was generated using a phospho starter pack.
This dataset contains a series of episodes recorded with a robot and multiple cameras. It can be directly used to train a policy using imitation learning. It's compatible with LeRobot and RLDS.
GEM4_pick_up_bottleobject_retrieval-preset-gemini
object_retrieval-preset-gemini
Real-robot teleoperation episodes of the instance_retrieval task on a single-arm Franka Research 3 cell (80 episodes, 36,239 frames at 10 fps,
released as Myungkyu/object_retrieval) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 10 frames = 1.0 s) and labels every sampled frame given only the
subtask preset of the task - the label list below… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/object_retrieval-preset-gemini.robot-meet-gemma-record-medicineThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 13,
"total_frames": 6119,
"total_tasks": 1,
"total_videos": 26,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 25,
"splits": {
"train": "0:13"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/AS-Robotics/robot-meet-gemma-record-medicine.Synchronised-Drumming-Gemini3Captionslayout_reconstruction-preset-gemini
layout_reconstruction-preset-gemini
Real-robot teleoperation episodes of the layout_reconstruction task on a single-arm Franka Research 3 cell (80 episodes, 37,559 frames at 10 fps,
released as Myungkyu/layout_reconstruction) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 10 frames = 1.0 s) and labels every sampled frame given only the
subtask preset of the task - the label list… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/layout_reconstruction-preset-gemini.RoboDojo-taco-gemini
RoboDojo-taco-gemini
RoboDojo long-horizon episodes (8 tasks, 800 episodes, 25 fps) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 25 frames) and labels every sampled frame under the
task-specific context (taco) induced for that task, in its hybrid form: labels name the target object's image coordinates only
where words cannot identify it (a random instance of a class that… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RoboDojo-taco-gemini.RoboDojo-preset-gemini
RoboDojo-preset-gemini
RoboDojo long-horizon episodes (8 tasks, 800 episodes, 25 fps) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 25 frames) and labels every sampled frame given only the
subtask preset of the task: the label list of the task-specific context (its hybrid form, so some labels carry a coordinate slot),
without the context's boundary criteria, sequence rule or… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RoboDojo-preset-gemini.pawbench-gemini-expansion-20260619
PAWBench Gemini Expansion 20260619
This dataset package contains the 5120-row PAWBench Gemini 3.5 Flash expansion input package for collaborator-side API evaluation.
Scope
Cosmos3 Nano, Table 1 PAWBench main only: 1560 videos.
Cosmos3 Super I2V, Table 1 PAWBench main only: 1560 videos.
Latest clean LTX2.3, Table 1 PAWBench main: 1560 videos.
Latest clean LTX2.3, K-scaling: 300 videos.
Latest clean LTX2.3, explicit endpoint: 140 videos.
LTX causal/cue… See the full description on the dataset page: https://huggingface.co/datasets/PhysEdit/pawbench-gemini-expansion-20260619.object_classification-preset-gemini
object_classification-preset-gemini
Real-robot teleoperation episodes of the object_classification task on a single-arm Franka Research 3 cell (80 episodes, 28,569 frames at 10 fps,
released as Myungkyu/object_classification) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 10 frames = 1.0 s) and labels every sampled frame given only the
subtask preset of the task - the label list… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/object_classification-preset-gemini.real_workbench-taco-gemini
real_workbench-taco-gemini
The four tasks of the real-robot Workbench Manipulation cell in one LeRobot v3.0 dataset with reviewed dense
high-level labels, pre-processed for π0.5 co-training: 320 episodes (4 x 80), 133,910 frames at 10 fps, single-arm
Franka Research 3, two camera views stored at 224x126, actions in the delta_eef space. Same frames as
Myungkyu/real_workbench, with a per-frame subtask label in
place of the task instruction.
This is the reviewed companion of… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/real_workbench-taco-gemini.real_workbench-taco-keyframe-gemini
real_workbench-taco-keyframe-gemini
Myungkyu/real_workbench-taco-gemini plus a
keyframe stream: the same 320 episodes (4 x 80), 133,910 frames at 10 fps, the same reviewed per-frame subtask labels,
the same 224x126 frames and delta_eef actions, with a third video feature observation.image.keyframe that carries the
Gemini-annotated visual memory. Everything below the keyframe section is identical to the parent dataset.
task_index
task
episodes
distinct subtask labels… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/real_workbench-taco-keyframe-gemini.test_0909_geminiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "grabette",
"total_episodes": 4,
"total_frames": 904,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 50,
"splits": {
"train": "0:4"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/simheo/test_0909_gemini.movement_reversal-preset-gemini
movement_reversal-preset-gemini
Real-robot teleoperation episodes of the movement_reversal task on a single-arm Franka Research 3 cell (80 episodes, 31,543 frames at 10 fps,
released as Myungkyu/movement_reversal) with dense high-level labels produced by the TACOR offline annotator:
Gemini 3.7 Flash reads each whole episode as one video clip (one sample every 10 frames = 1.0 s) and labels every sampled frame given only the
subtask preset of the task - the label list below… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/movement_reversal-preset-gemini.GEM4_Hold_down_the_cardboard_boxthanos_picking_power_gemThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 51,
"total_frames": 16267,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:51"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/mishig/thanos_picking_power_gem.RoboDojo-taco-visual2-gemini
RoboDojo-taco-visual2-gemini
visual2 variant (2026-09-14): unlike RoboDojo-taco-visual-gemini (marker in a separate keyframe video), here the point marker (red disc, radius 1.9 % of the width, white ring) is drawn into the head camera video itself for every frame whose governing label names a position, and the label text says ... the location marked with the red dot ... (tic-tac-toe: ... <cell> marked with the red dot.). The dataset keeps the three live views only (no keyframe… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/RoboDojo-taco-visual2-gemini.GEM-250K
GEM: Generative Supervision Helps Embodied Intelligence
Ruowen Zhao1,
Bangguo Li1,
Zuyan Liu1,2,†,
Yinan Liang1,
Junliang Ye1,
Fangfu Liu1,
Diankun Wu1,
Zhengyi Wang1,
Xumin Yu2,
Yongming Rao2,✉,
Han Hu2,
Jun Zhu1,✉
†Project Lead.✉Corresponding Author.
1Tsinghua University,
2Tencent Hunyuan
Abstract
Embodied Vision-Language Models (VLMs) have demonstrated impressive… See the full description on the dataset page: https://huggingface.co/datasets/zzzrw/GEM-250K.gemma_rover_drop_dirt_up_to_5
gemma_rover_drop_dirt_up_to_5
This dataset was generated using a phospho starter pack.
This dataset contains a series of episodes recorded with a robot and multiple cameras. It can be directly used to train a policy using imitation learning. It's compatible with LeRobot and RLDS.
test_geminiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "grabette",
"total_episodes": 1,
"total_frames": 579,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 50,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/SteveNguyen/test_gemini.object_retrieval-taco-gemini
object_retrieval-taco-gemini
Real-robot teleoperation episodes of the Object Identification task (internal id instance_retrieval) on a single-arm Franka Research 3
cell (80 episodes, 36,239 frames at 10 fps, released as Myungkyu/object_retrieval)
with dense high-level labels and visual memory produced by the TACOR offline annotator: Gemini 3.7 Flash reads each whole episode as one
video clip (one sample every 10 frames = 1.0 s) and labels every sampled frame given the full task… See the full description on the dataset page: https://huggingface.co/datasets/Myungkyu/object_retrieval-taco-gemini.gemma_rover_scoop_up_to_4
gemma_rover_scoop_up_to_4
This dataset was generated using a phospho starter pack.
This dataset contains a series of episodes recorded with a robot and multiple cameras. It can be directly used to train a policy using imitation learning. It's compatible with LeRobot and RLDS.