GalaxyGeneralRobotics/HumanTracker

Dataset Card for HumanTracker

Project page · Paper · Code

HumanTracker is a humanoid motion-tracking benchmark. This release contains two complementary subsets:

• —`motions/` — the evaluation test split: retargeted 29-DoF reference trajectories, grouped into four motion families.
• —`preference_pair/` — 6,000 human preference pairs, each stored with the two tracker rollouts that were compared and the source-motion clip they track.

The evaluation harness and HumanScore reward model live in the HumanTracker repository. preference_pair/ is the reward model's training input as published: the rollouts are inline, so nothing has to be re-simulated to reproduce HumanScore.

Dataset Details

Humanoid tracking is often scored with per-frame kinematic error, which misses the physical artifacts people notice in video — unstable support, foot skating, mistimed contacts. HumanTracker pairs a large, family-labeled motion test set with a preference-aligned metric (HumanScore) trained on pairwise human comparisons.

Motions are retargeted to a 29-DoF Unitree G1-style humanoid with GMR and stored as qpos trajectories at 50 Hz. Preference pairs compare GMT, TWIST2, SONIC and Humanoid-GPT rollouts of the same reference window (typically 250 frames / 5 s). Labels are a strict preference, similar, or bad_traj (cannot compare). The pair split is grouped by motion_id, so every clip from one source motion stays in one partition.

Paper: HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark (ECCV 2026).

License: Apache 2.0.

Dataset Structure

HumanTracker/
  README.md
  motions/
    test.json
    Daily/
    Ground/
    HighlyDynamic/
    Interaction/
  preference_pair/
    train.json
    test.json
    train/train-00000-of-00020.parquet ... train-00019-of-00020.parquet
    test/test-00000-of-00005.parquet  ... test-00004-of-00005.parquet

Filenames are anonymized for release. Dates, performer names, capture-system tags and sample-rate suffixes are removed. Family-level names (Daily, Interaction, HighlyDynamic) are numbered (Daily_1.npz). Action labels that are themselves the motion type are kept: Ground actions such as burpee and sit-lie, and Highly Dynamic actions such as Tennis or named martial-arts skills.

Motions (motions/)

motions/test.json is a list of

{"path": "Daily/Daily_1.npz", "category": "Daily", "frames": 1584}

Each .npz contains:

The evaluator in the code repository reads the same manifest:

from pathlib import Path
import json
import numpy as np

root = Path("motions")
items = json.loads((root / "test.json").read_text())
item = items[0]
traj = np.load(root / item["path"])
qpos = traj["qpos"]          # (frames, 36)
category = item["category"]  # Daily | Ground | HighlyDynamic | Interaction

python -m humantracker.eval.eval_parallel_tracker \
    --tracker sonic \
    --mocap_path /path/to/HumanTracker/motions \
    --test_json /path/to/HumanTracker/motions/test.json \
    --termination_metric whole_body

path is relative to motions/. The first path component must match category.

Preference pairs (preference_pair/)

Load with 🤗 Datasets:

from datasets import load_dataset

ds = load_dataset("GalaxyGeneralRobotics/HumanTracker", name="preference")
row = ds["train"][0]
print(row["choice_type"], row["tracker_pair_key"], row["motion_id"])

Or read the parquet shards directly, which is what the reward-model trainer does:

import io
import json
import numpy as np
import pyarrow.parquet as pq

table = pq.read_table("preference_pair/test/test-00000-of-00005.parquet")
row = table.slice(0, 1).to_pylist()[0]

annotation = json.loads(row["annotation_json"])
reference = np.load(io.BytesIO(row["motion_npz"]))        # same keys as motions/*.npz
candidate_0 = np.load(io.BytesIO(row["candidate_0_npz"]))
candidate_1 = np.load(io.BytesIO(row["candidate_1_npz"]))
print(row["choice_type"], row["preferred_candidate_idx"], row["candidate_0_tracker"])
print(candidate_0["joint_pos"].shape)                     # (num_frames, 29)

Candidate slots are stable identities, not display positions: preferred_candidate_idx indexes them, and the order the annotator saw is recorded separately in annotation_json. motion_npz carries the same keys as motions/*.npz, already sliced to [source_start_frame, source_end_frame). Most windows are 250 frames (5 s at 50 Hz); shorter tail windows are kept and right-padded at training time.

Each candidate NPZ is one tracker's closed-loop rollout of that window, frame-aligned with the reference, float32, num_frames rows per array:

The reported HumanScore model concatenates the first two blocks into a 539-d per-frame token; the residual block is shipped for the paper's appendix ablation and is unused by default. qpos / qvel are the MuJoCo generalized state, for replaying a rollout in the viewer.

train.json and test.json list the record_ids of each split, grouped by motion_id. train.json also carries model_selection, the 461 records held out for epoch selection, so a rerun selects the same checkpoint as the published one. bad_traj pairs are excluded from the fit, leaving 5,757 trained pairs; preference uses a Bradley–Terry loss and similar a symmetric 0.5 target.

The six unordered tracker pairs (gmt|hgpt, gmt|sonic, gmt|twist2, hgpt|sonic, hgpt|twist2, sonic|twist2) are balanced at 1,000 pairs each.

Training HumanScore from this directory:

python -m humantracker.reward_model.train.trainer \
    --data_dir /path/to/HumanTracker/preference_pair \
    --cache_dir /path/to/feature_cache \
    --output_dir storage/checkpoints/reward_model

Uses

• —Tracker evaluation. Run a policy on motions/ with the published evaluator and report Succ / MPJPE / HumanScore per family.
• —Reward-model / HumanScore research. Reproduce or extend HumanScore directly from preference_pair/; the code repository reads this directory as its --data_dir.
• —Diagnostics. Family labels and retained action names (burpee, Tennis, …) support fine-grained error breakdowns.

This release is not a full training-motion dump. The 2,500 evaluation clips are the official test split; preference clips are the labeled 5 s windows, not the complete source takes.

Citation

@misc{liu2026humantrackercomprehensivehumanalignedmotion,
      title={HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark},
      author={Dairu Liu and Zekun Qi and Jiayu Zeng and Ruixi Yu and Yu Guan and Yintianrun Zhang and Xuchuan Chen and Sikai Liang and Zekai Li and Chenghuai Lin and Xinqiang Yu and Wenyao Zhang and He Wang and Li Yi},
      year={2026},
      eprint={2608.13555},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2608.13555},
}