CK-Sung/gr00t-n17-g1-dex3-nubjuki-abs

GR00T N1.7 — G1 + Dex3, all-ABSOLUTE action ablation

This is an ablation control, not the recommended model. It exists to answer one question: does the action representation matter for this task? It does — the relative-arm model is 26 % better and should be used instead.

Everything here is identical to that model except the action representation: same dataset, same 90/10 split, same 10k steps, same effective batch 64, same 16-step chunk at 30 fps, same 3 cameras, same 4×A100 setup. Only left_arm / right_arm changed from RELATIVE to ABSOLUTE (hands were already ABSOLUTE in both).

• —Dataset: `carroll511/g1_dex3_nubjuki` — 100 episodes / 31,803 frames / 30 fps (17.7 min), single task
• —Checkpoint: step 8,000 (best of 2k–10k on the held-out 10-episode val split)

Result: RELATIVE arms clearly win

Val open-loop MAE in radians — directly comparable because both models return absolute joint targets after decoding.

The hands are ABSOLUTE in both configurations and score nearly the same, which is the internal control that makes this comparison trustworthy: the gap comes from the arms alone.

The effect is largest on the left arm, which barely moves during this task — a relative model only has to predict "no change", while an absolute model must regress the exact pose every step. The right hand, the one actually grasping, is the one group where absolute is not worse; its error is dominated by fine finger motion rather than by pose drift.

Checkpoint sweep (this model)

Unlike the relative model (which was still improving at 10k), this one peaks at 8k and edges back up — a mild sign that the absolute parameterisation starts overfitting sooner on 17.7 min of data.

Open-loop error is a proxy; closed-loop evaluation on the robot is the real test.

Interface

Identical to the relative model except that the policy predicts absolute joint targets directly (no delta-to-absolute reconstruction step). Observation: 3 RGB cameras 480×640 (ego_view ← head, right_wrist_view, left_wrist_view) + current 28-d joint state (radians) + the exact training string "pick up the doll and place it in the red region.". Action: 28-d joint targets, 16-step chunk intended for 30 Hz execution.

Joint order (same for state and action): left_arm 0–6, right_arm 7–13, left_hand 14–20, right_hand 21–27.

⚠️ The two hands do not share a finger order: left is Thumb0-2 → Middle0,1 → Index0,1, right is Thumb0-2 → Index0,1 → Middle0,1, and their bending signs are mirrored. This comes from the source dataset's metadata and the model learned it as-is.

python gr00t/eval/run_gr00t_server.py --model-path <this repo> --embodiment-tag new_embodiment

Isaac-GR00T (validated at 65cc4a192), transformers 4.57.3, torch 2.7.1+cu128, and access to the gated nvidia/Cosmos-Reason2-2B tokenizer are required. The VLM backbone is frozen, so deployed camera placement/exposure should match the training data.