mncai/G1_Dex3_Trash_LocoManipulation

G1 Dex3 Trash Loco-Manipulation

Teleoperated demonstrations of a Unitree G1 humanoid with Dex3-1 hands picking a plastic trash bag off the floor and putting it into a bin. Whole-body loco-manipulation: walking, kneeling, grasping, standing back up and turning.

118 episodes, 183,000 frames, about 61 minutes at 50 Hz.

<img src="assets/normal.gif" width="480" alt="one full episode, ego view, 4x speed">

One episode from the head camera, 4x speed. Walk, kneel, grasp, stand, turn, carry, drop.

The 64-dimensional `action.motion_token` is the output of the GEAR-SONIC v1.1 encoder. Its latent space is specific to that checkpoint. Tokens will not decode correctly with sonic_release or low_latency. To replay them you need sonic_v1_1/model_decoder.onnx from nvidia/GEAR-SONIC. The 43-dimensional joint states are plain joint angles and have no such dependency.

Task

0  walk to the trash bag
1  pick up the trash bag
2  turn around
3  carry it to the bin
4  put it in the bin

The robot starts standing several steps away, walks up, kneels, grasps the bag with the left hand, stands, turns roughly 180 degrees, walks to the bin and drops the bag in. Only the left hand is used. The right hand stays open for the whole dataset.

Contents

118 episodes   183,000 frames   61 minutes
  0  -  80     normal demonstrations
  81 - 117     failure and recovery demonstrations

Episodes 81 to 117 are recovery demonstrations. In each one the operator deliberately fails the grasp, then recovers and completes the task. The failed attempt is real data and a policy trained on it without masking will learn the failure, roughly 6.7% of all frames. Cut them out or weight them down if that is not what you want.

Format

LeRobot v2.1. Video is H.264, 480x640, 50 fps.

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
ds = LeRobotDataset("mncai/G1_Dex3_Trash_LocoManipulation")

Observations

Camera

Intel RealSense D435i, color stream only. Depth is not recorded.

sensor         640x480 color at 30 fps
stored as      480x640x3, H.264, yuv420p
container fps  50
mount          head, torso frame (0.06, 0.00, 0.45) m, pitched 47.6 degrees down

Joint order for the 43-dimensional state:

0:6    left leg          6:12   right leg        12:15  waist
15:22  left arm          22:29  left hand        29:36  right arm     36:43  right hand

Each Dex3-1 hand has 7 joints: thumb 0/1/2, middle 0/1, index 0/1.

Actions

Collection

Teleoperated on a real Unitree G1 through a PICO VR rig, three-point tracking, with GEAR-SONIC v1.1 as the whole-body controller. The operator drives the upper body directly while the controller handles balance and locomotion. Every frame in this dataset is real hardware. There is no simulation and no retargeted motion capture.

Intended use

Post-training a vision-language-action model for humanoid loco-manipulation. The dataset was built for GR00T N1.7 with the UNITREE_G1_SONIC embodiment, but the joint states and video are usable with any policy that accepts a 43-dimensional humanoid.

License

Apache License 2.0. See LICENSE.

The video, joint states and annotations are original recordings. The 64-dimensional motion tokens are outputs of the GEAR-SONIC model. Under the NVIDIA Open Model License NVIDIA makes no ownership claim to model outputs.

Built with NVIDIA GEAR-SONIC and NVIDIA Isaac GR00T.

Citation

@misc{g1_dex3_trash_locomanipulation,
  title  = {G1 Dex3 Trash Loco-Manipulation},
  year   = {2026},
  url    = {https://huggingface.co/datasets/mncai/G1_Dex3_Trash_LocoManipulation}
}