Datasetpublic

rooty2020/RB-Y1_WujiHand2_teleop_ego_exo_100_pointtracks

RB-Y1 + WujiHand2 — ego+exo 3D robot & hand point tracks 3D point tracks for 100 teleop episodes of RB-Y1 with WujiHand2 dexterous hands, ego (ZED-M) + exo (ZED 2i) views. Resampled to 15 Hz (22758 frames). Per episode (episode_XXX/) file contents robot.npz arm points (T,512,3), robot base frame hand.npz WujiHand2 hand points (T,256,3), robot base frame ego.mp4,exo.mp4 the two views, 15 fps, intra-only (-g 1) action.parquet actions @15 Hz… See the full description on the dataset page: https://huggingface.co/datasets/rooty2020/RB-Y1_WujiHand2_teleop_ego_exo_100_pointtracks.

sourceHugging Facecc-by-nc-4.0updated 7d agoView on Hugging Face

0likes37downloads

Dataset Card

RB-Y1 + WujiHand2 — ego+exo 3D robot & hand point tracks

3D point tracks for 100 teleop episodes of RB-Y1 with WujiHand2 dexterous hands, ego (ZED-M) + exo (ZED 2i) views. Resampled to 15 Hz (22758 frames).

Per episode (`episode_XXX/`)

file	contents
`robot.npz`	arm points `(T,512,3)`, robot base frame
`hand.npz`	WujiHand2 hand points `(T,256,3)`, robot base frame
`ego.mp4`,`exo.mp4`	the two views, 15 fps, intra-only (`-g 1`)
`action.parquet`	actions @15 Hz
`meta.json`	ego+exo intrinsics + `T_ego_base`/`T_exo_base`, per-frame ego pose

Projection

Points are in the robot base frame. To a camera: p_cam = R @ p_base + t with meta.cameras.<cam>.T_<cam>_base, then u=fx*x/z+cx, v=fy*y/z+cy.

Differences from the usual bwjhdataset format (DROID / RoboCasa / trellis)

This dataset intentionally deviates from our standard convention — read before plugging into the usual loader:

aspect	usual bw_jh format	this dataset
gripper/hand file	`gripper.npz` (2F gripper)	`hand.npz` (WujiHand2 dexterous hand, 256)
video names	`view1.mp4`, `view2.mp4` (both exterior)	`ego.mp4` (head-mounted, moving) + `exo.mp4` (external, static)
point frame	per-frame view1 camera frame (`cam_from_world`, points already in-camera)	robot base frame (project via `T_ego_base`/`T_exo_base` in meta)
tar layout	shard tars of `<id>/`	one tar of `episode_XXX/`

So it is NOT drop-in for a loader that assumes points are already in the view1 camera frame — apply meta.cameras.<cam>.T_<cam>_base first. Ask if you want a re-export in the exact usual format (view1/view2, gripper.npz, per-frame view1-cam-frame cam_from_world).

Notes

—FK from observation.joint_position through the source meta/robot_model MJCF; cameras from camera_calibration.json.
—EXO projection is exact; EGO has a small far-left residual = the dataset's own manual ego extrinsic (IoU~0.8), not a pipeline issue.
—Original data 30 Hz → shipped 15 Hz (every 2nd frame, consistent across points/videos/actions).
—Source: kdh8156/RB-Y1_WujiHand2_teleop_ego_exo_100.