CoolFace
Datasetpublic

kimtaey/robotwin2.0-egowam-flow-cam_high-subset

RoboTwin 2.0 cam_high EgoWAM-style 3D Flow — snapshot subset (8,522 episodes) Query-based 3D motion-flow sidecar labels for the RoboTwin 2.0 LeRobot dataset (yuanty/robotwin2.0-fastwam), generated with a pretrained 3D point tracker (Track4World, DA3 backbone, metric-scale mode) from RGB only — EgoWAM-style (arXiv 2607.08436 §4.2 conventions). This is a frozen snapshot of an in-progress full-dataset run (27,500 episodes); see snapshot_manifest.json for the exact episode list and… See the full description on the dataset page: https://huggingface.co/datasets/kimtaey/robotwin2.0-egowam-flow-cam_high-subset.

sourceHugging Faceotherupdated 2mo agoView on Hugging Face
0likes1.9kdownloads
Dataset Card

RoboTwin 2.0 cam_high EgoWAM-style 3D Flow — snapshot subset (8,522 episodes)

Query-based 3D motion-flow sidecar labels for the RoboTwin 2.0 LeRobot dataset (yuanty/robotwin2.0-fastwam), generated with a pretrained 3D point tracker (Track4World, DA3 backbone, metric-scale mode) from RGB only — EgoWAM-style (arXiv 2607.08436 §4.2 conventions). This is a frozen snapshot of an in-progress full-dataset run (27,500 episodes); see snapshot_manifest.json for the exact episode list and generation parameters.

Contents

  • —episodes/<bucket>/episode_XXXXXX.npz + .meta.json — one pair per episode, sharded into subdirectories by bucket = episode_index // 1000 (zero-padded to 3 digits), e.g. episode 6819 → episodes/006/episode_006819.npz. Sharding keeps every directory under the Hugging Face 10,000-files-per-directory limit.
  • —snapshot_manifest.json — frozen episode list + generation parameters
  • —analysis.json — full analysis (per-task-block coverage, stats below)

Per-episode NPZ schema

keyshapedtypemeaning
flow_xyz[T, 8, 1120, 3]f16/f32*3D displacement per query per horizon, source-frame camera coords (OpenCV axes: x right, y down, z forward), nominally meters
valid[T, 8, 1120]boolin-episode & finite & unmasked source pixel (invalid ≠ zero-motion)
visible[T, 8, 1120]f16tracker visibility in [0,1] (raw)
confidence[T, 8, 1120]f16tracker confidence in [0,1]
query_uv[1120, 2]f3228x40 uniform grid, pixel coords on 640x480, row-major: u=(c+.5)W/40, v=(r+.5)H/28
query_uv_tracker[1120, 2]f32same grid at tracker res (640x448)
traj_uv[T, 8, 1120, 2]f16tracked 2D position per horizon (tracker res)
future_offsets[8]i32[4,8,12,16,20,24,28,32] frames @ 50 fps = 0.08–0.64 s
frame_index[T]i32source-frame index into the LeRobot episode
pose_drift[T]f32max abs tracker-c2w drift per window (QA; camera is fixed)

\* dtype mix in this snapshot: ~75% float16, ~25% float32 (early episodes; values identical semantics). flow = X_fut_src − X_src with X_fut_src = inv(c2w_t) @ c2w_fut @ X_fut (EgoWAM's exact formula). One tracker call per source frame over the window [t, t+32] (first-frame dense mode); the 28x40 grid is sampled nearest-neighbor from the dense output.

Snapshot statistics

  • —Episodes: 8,522 covering 50/50 task blocks of RoboTwin 2.0 (block = episode_index // 550); split: 1,317 clean + 7,205 randomized.
  • —Embodiment: aloha dual-arm (14-dim state/action: 2× waist, shoulder, elbow, forearmroll, wristangle, wristrotate, gripper); camera `camhigh` 640x480@50fps, fixed within episodes (verified against simulator GT poses).
  • —Episode length (frames): min 74 / p25 115 / p50 152 / p75 277 / max 1037; 1.74M source frames total.
  • —Quality (160-episode sample): valid ratio 0.872, visibility 0.817, confidence 0.850; median per-episode |flow| p50 ≈ 2.9 mm, p99 ≈ 0.157 m. All shapes verified [T,8,1120,3]; every uploaded npz CRC-validated.
  • —Size: 145.5 GB total (~17 MB median per episode).

Loading

python
import numpy as np
from huggingface_hub import hf_hub_download

def load_episode(i, repo="kimtaey/robotwin2.0-egowam-flow-cam_high-subset"):
    bucket = f"{i//1000:03d}"
    npz = hf_hub_download(repo, f"episodes/{bucket}/episode_{i:06d}.npz",
                          repo_type="dataset")
    return np.load(npz)

Provenance & caveats

  • —Tracker: Track4World track4world_da3.pth (TencentARC), --metric_scale: output is nominally metric (DA3 metric depth); per-episode scale varies ~±15% — treat cross-episode magnitudes accordingly (scale_mode: metric_nominal_da3 in each meta.json). Pilot validation: static-background 3D noise floor p50 ≈ 4 mm; tracker pose drift ≤ 0.03 on the fixed camera.
  • —RGB-only: no simulator state, depth GT, or segmentation used.
  • —Generation code: tools/robotwin_3dflow in the robotic-representation-models repo (branch agent/skt).
  • —Episode 344 is intentionally absent (a corrupted write was detected by CRC audit, purged, and queued for regeneration; it will return in the full-set release).