Modelpublic

prehj/GR00T-N1.5-dexjoco-single-arm-baseline

sourceHugging Faceapache-2.0updated 28d agoView on Hugging Face

0likes28downloads

Model Card

GR00T-N1.5-dexjoco-single-arm-baseline

GR00T-N1.5 finetuned on the DexJoCo single-arm multitask set. This is the reference (baseline, non-MoE) policy for the DexJoCo arm of the quantizability gate project.

—Benchmark: DexJoCo, single-arm multitask. The single-arm task set is water_plant, hammer_nail, pick_bucket, pinch_tongs, fold_glasses, click_mouse — 6 tasks, 100 episodes each.
—Embodiment: single arm with a 16-DoF dexterous hand. Embodiment tag in the shipped metadata is new_embodiment.
—Observations: two 640x640 RGB views — front, wrist — at 30 fps
—State: arm position (3) + arm rotation (quaternion, 4) + hand joints (16)
—Actions: arm position (3) + arm rotation (3) + hand joints (16), all absolute, action horizon 16
—Finetuning data: dexjoco_lerobot_datasets, LeRobot v3.0 packed layout
—Checkpoint: step 60,000 = 17.53 epochs (configured for 18 epochs; 60k steps came first). Final training loss 0.0069, from 1.084 at step 0.

A caution for anyone porting the gate here

The action space is absolute, not delta. Risk descriptors written for RoboCasa's delta-action space do not transfer: applied unchanged to DexJoCo the computed risk flags fired on 87% and 95% of chunks — noise, not signal. Any gate for this embodiment needs its descriptors and thresholds re-derived from these actions.

There is also a dual-arm DexJoCo baseline at the same recipe which is not published here.

Architecture

GR00T-N1.5 (model_type: gr00t_n1_5, architectures: ["GR00T_N1_5"]), unchanged from the NVIDIA release:


VLM backbone	Eagle — `NVEagle/eagle_er-qwen3_1_7B-Siglip2_400M_stage1_5_128gpu_er_v7_1mlp_nops`
features tapped at	`select_layer: 12`, backbone embedding dim 2048
backbone tuning	`tune_llm: false`, `tune_visual: true` (vision tower trained, LLM frozen)
action head	flow-matching DiT, 16 layers, 32 heads, head dim 48, cross-attn dim 2048, interleaved self-attention, `ada_norm`, dropout 0.2
VL self-attention	4 layers, 32 heads, head dim 64
action head tuning	`tune_diffusion_model: true`, `tune_projector: true`
action space	`action_dim: 32` (padded), `max_state_dim: 64`
action horizon	16
inference denoising steps	4
dtypes	`model_dtype: float32`, `compute_dtype`/`torch_dtype: bfloat16`

Training recipe

Read directly out of this checkpoint's trainer_state.json and the run's training_args.bin:


steps	60,000 (this is the final checkpoint; `max_steps == global_step`)
batch size	64 effective — 32 per device x 2 GPUs, no gradient accumulation
learning rate	1e-4, cosine schedule, warmup ratio 0.05
weight decay	1e-5
precision	bf16
epochs	17.53 (60k steps reached first; `num_train_epochs: 18`)
dataloader workers	16
final loss	0.0069

How to load and serve

Requires NVIDIA Isaac-GR00T and transformers==4.51.3 (the version this checkpoint was written with).

bash

huggingface-cli download prehj/GR00T-N1.5-dexjoco-single-arm-baseline --local-dir ./ckpt

python

from gr00t.model.policy import Gr00tPolicy
from gr00t.experiment.data_config import DATA_CONFIG_MAP

data_config = DATA_CONFIG_MAP["<your dexjoco data config>"]
policy = Gr00tPolicy(
    model_path="./ckpt",
    modality_config=data_config.modality_config(),
    modality_transform=data_config.transform(),
    embodiment_tag="new_embodiment",
    device="cuda",
)
action_chunk = policy.get_action(observation)   # 16 steps

Or serve it over the Isaac-GR00T inference service and drive it from the simulator client:

bash

python scripts/inference_service.py --server \
  --model_path ./ckpt --embodiment_tag new_embodiment --denoising_steps 4

experiment_cfg/metadata.json ships in this repo and carries the state/action normalisation statistics and the modality schema — the policy will not produce correct actions without it, so keep the directory layout intact.

Files

file	what it is
`model-0000{1,2}-of-00002.safetensors`, `model.safetensors.index.json`	the weights (~7.1 GB)
`config.json`	GR00T-N1.5 architecture config
`experiment_cfg/metadata.json`	embodiment schema + normalisation statistics
`trainer_state.json`	full 60k-step loss history