hooneyskywalker/g1-motion-tracking-policies
G1 motion tracking policies
Code: github.com/hooneyskywalker0127/g1-motion-tracking Videos: YouTube playlist
Six seconds of walk2_subject4. The trained policy on the left, the reference it is following on the right, both in Isaac Lab.
Whole-body control policies that make a Unitree G1 humanoid follow human motion capture clips in simulation. One policy per motion, plus a single distilled policy that covers fourteen of them.
Why they exist: while building a swarm humanoid simulation, GEAR-SONIC from NVIDIA's GR00T Whole-Body Control was used to drive the G1. Using a controller and building one teach different things, so this pipeline was built end to end. SONIC also tracks a reference motion, so this is the same problem solved by a different hand.
The pipeline behind them: human mocap → retargeting to the G1 → reference motion → policy training. Stages 1 to 3 have no physics; physics enters at stage 4, where the robot has to hold itself up.
What is here
policies/<sequence>/ holds one trained policy each.
model_29999.pt— RSL-RL checkpoint at (30,000) iterationspolicy.onnx— the actor network exported for inference, 160 in and 29 outrun_name.txt— the training run folder the checkpoint came from
eval/<sequence>.json holds the measured numbers for that policy.
How to run a policy
policy.onnx is the actor network on its own. It takes one observation vector and returns one action. It holds no reference motion, so the caller supplies the reference terms.
The observation is 160 values concatenated in this order.
The action is an offset on the joint position target: target = default joint position + action, for all (29) joints. Control runs at (50) Hz (sim step (0.005) s, decimation (4)), the same rate as the reference motion.
import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("policies/walk1_subject1/policy.onnx")
obs = np.zeros((1, 160), dtype=np.float32) # fill as in the table above
action = sess.run(["actions"], {"obs": obs})[0] # (1, 29)Joint order is the articulation order of the G1 asset used by BeyondMimic. Read it from the environment rather than assuming one.
How they were trained
BeyondMimic on Isaac Sim (5.1) and Isaac Lab (2.3.2), PPO, (4,096) environments, (30,000) iterations, one policy per motion. The reference motions come from LAFAN1 retargeted to the G1 with GMR.
Reward is tracking accuracy against the reference, not gait plausibility.
Results
Measured over (100) rollouts with domain randomization off. Completion rate is the share of rollouts that reach the end of the clip without the anchor body crossing a height or orientation threshold. The error column is the mean global body position error over all rollouts.
Three sequences complete no rollout. walk3_subject4 and walk3_subject1 track most of the clip before the anchor crosses the threshold, at 88% and 78% of the frames. obstacles2_subject1 is the outlier at 18%.
obstacles2_subject1 fails on purpose to be informative: that clip has the actor climbing stairs, with the pelvis above (1.05 m) for (48.7) seconds. The training ground is flat, so the reference is not reachable and the error never drops. It is kept here as a record of what the selection criterion missed, since the criterion only looked at retargeting foot error and never asked whether the target was physically possible on flat ground.
One policy for fourteen sequences
student/final_model.pt is a single policy that tracks fourteen of the sequences above. It was distilled from the fourteen per-clip policies by DAgger: the student rolls out, the matching teacher labels the states it visits, and the student regresses onto those labels.
Measured the same way as the table above, domain randomization off, over (64) rollouts per sequence.
Mean global body position error is (90) mm against the teachers' (107) mm on the same fourteen. Compressing fourteen policies into one did not cost accuracy here.
The three sequences that complete no rollout are not in the set, and neither is kobe_level1, which is held out.
What it took
Most of the work was one sequence. aiming1_subject1 sat at 0% while the other thirteen were at 100%, and every fix moved the failure to a different clip instead of removing it. Two changes mattered.
Training episode length, (10) s to (40) s. This is the larger of the two. On an eight-clip subset, with everything else held fixed over (50,000) iterations:
Evaluation asks the policy to hold on for the whole clip, up to (261) s. Training only ever asked for (10) s. aiming1_subject1 fell at (64) s, past any window it had been trained through. Adding noise to the rollout, which widens the state distribution, did nothing.
Network size, four times. Episode length alone left the fourteen-clip set at a mean of 97.5% with one clip in the seventies, and the clip that sat there changed with every other lever tried — more iterations, failure-driven sampling, per-clip loss balancing. Two different levers landed on the same 97.5%. Raising the hidden layers from (1024, 1024, 512, 512, 256) to (2048, 2048, 1024, 1024, 512) took that to 99.9% with no clip lower than 98%, and nothing regressed.
What was ruled out along the way, by measurement: the failing clip's own teacher (96.9%, and the student tracks it as accurately as the teacher does), gradient conflict between clips (per-clip gradients all point with the aggregate; pairwise conflict ratio 12.3%), over-training (the laggard converges last, not first), and minibatch starvation of the failing clip (a run where its share collapsed to 1.1% still completed every rollout).
What is in student/
final_model.pt— the distilled policy, (50,000) iterationsconfig.json— every hyperparameter, including the fourteen teacher runseval_14.json— the measured numbers aboverun_name.txt— the training run folder
The observation is (260) values and the action is (29). Unlike the per-clip policies, the action is a residual on the reference joint positions rather than on the default pose, and the velocity terms in the observation are scaled by (0.05). Both are recorded in config.json and applied by the loader in the code repository. There is no onnx export for this one yet.
sim-to-sim — does it survive a second simulator
A policy that only works in the simulator it was trained in is not evidence of anything. The same onnx actor was loaded into MuJoCo with no retraining and no fine-tuning, and all seventeen sequences were replayed for the full clip length.
Six seconds of walk2_subject4. Isaac Lab on the left, the same policy in MuJoCo on the right, both panels on the same instant.
two_standards.txt has the per-sequence numbers under both pass criteria on the matched protocol, and sim2sim_table.png renders the same tables. sym/<sequence>_simdr.json holds the Isaac side of each.
There is no single pass criterion
BeyondMimic scores by termination. The episode ends the moment any of three conditions fires, and all three look at z alone.
PolySim counts a rollout failed once the mean global body position error crosses 0.5 m. None of the termination conditions sees horizontal drift; PolySim's threshold is built to catch exactly that.
One caveat. PolySim's text says mean body position error over 0.5 m, but the released code tests whether any single body exceeds a curriculum threshold (1.5 m by default) and disables that check in the default configuration. The numbers below implement the text.
Is anything lost in transfer
The same policy was run in Isaac, where it was trained, and in MuJoCo, which it had never seen. The numbers do not drop.
100 runs each. Error is averaged over completing runs only, so the three that complete none (obstacles2_subject1, walk3_subject1, walk3_subject4) drop out of the last three rows.
The most dynamic five seconds of dance2_subject3. Isaac Lab on the left, the same policy dropped into MuJoCo on the right. This sequence is where the two simulators agree most closely — 0.99 against 1.00 completion, 104.4 against 104.3 mm global error.
The rest of this section is how those numbers were produced.
Putting two simulators side by side requires the two columns to be the same quantity. Four things were matched: one scorer for both sides, the same initial perturbation distribution, reference and robot read at the same instant, and no early termination on either side with both criteria computed afterwards.
That last one matters most. With Isaac's termination enabled an episode ends the moment the robot falls, so global error after the fall is never recorded and a failed rollout scores as a PolySim success. MuJoCo has no termination, rolls to the end, and a fallen robot always crosses 0.5 m. The two numbers would carry the same name while measuring different things.
Three conditions: sim is Isaac without perturbation, sim-dr is Isaac with it over 100 environments, sim2sim is MuJoCo with the same perturbation over 100 trials. The window is full clip length.
Retention is MuJoCo/Isaac for success and Isaac/MuJoCo for error, so 100 % means nothing was lost either way. Nothing is lost.
Above 100 % does not mean MuJoCo is the better engine. Contact handling and the solver differ. The sentence this supports is that there is no transfer loss, and no more. The comparable published figure is PHUMA appendix D.3, 90.5 % and 93.2 % going from Isaac Gym to MuJoCo.
Per sequence, ordered by completion rate.
Isaac columns are sim-dr (100 environments), MuJoCo columns are sim2sim (100 trials). Errors are averaged over completing trials only, so the three with none are undefined.
The seventeen fall into four groups by motion type.
The ten walking, aiming and dance clips hold Sbm at 0.94 or better on both sides and Spoly at 0.86 or better, with 61-115 mm global and 31-45 mm local error. All four metrics sit close together across the two simulators.
Running and jumping (run2_subject4, jumps1_subject1) hold Sbm at 0.73-0.98 while Spoly drops to 0.00-0.14. They do not fail by falling, they fail by drifting, and their global error of 155-181 mm is the largest of the seventeen. One criterion alone hides this entirely.
obstacles3_subject3 and walk2_subject3 go the other way on Sbm: 0.58 and 0.64 in Isaac against 0.80 and 0.76 in MuJoCo. `walk2subject3` is the one exception worth naming — its S_poly falls from 0.45 to 0.00, the single cell where MuJoCo is clearly worse.
The three with no completed rollout have no error to report. obstacles2_subject1 survives 8.6 % of its clip and is unconverged; walk3_subject1 and walk3_subject4 reach 77-87 % and fail near the end.
Three things run through all of it. MuJoCo is worse than Isaac in only three cells out of the fourteen that report error. Error magnitude is set by motion difficulty rather than by simulator: 61-115 mm for walking, 104-166 mm for obstacles and dance, 155-181 mm for jumping and running. And failure splits in two: a low Sbm means the robot fell, while a low Spoly alone means heading error accumulated over a long clip.
The two scorers were checked against each other first. Isaac's env 0 rollout was dumped in full, rescored with the MuJoCo scorer and compared against the online values: all five metrics agree to four decimal places, the residual coming from the dump being float16.
Posture crosses over, position does not
Joint angle error is 0.594 rad in Isaac and 0.593 rad in MuJoCo. Posture crosses over with essentially no loss.
What does not cross over is position. Re-anchoring MuJoCo's 101.3 mm global body error to the robot anchor drops it to 38.0 mm, so 62 % of the error is root position and heading drift rather than posture. The same drift appears in Isaac (108.3 to 40.6 mm), so it is not a MuJoCo artefact.
The three that do not complete
Fourteen of seventeen complete the full clip. The other three are the zeros above, and they differ in kind: obstacles2_subject1 survives 8.6 % of its clip and is unconverged, while walk3_subject1 and walk3_subject4 reach 77-87 % and fail near the end, where the LAFAN1 actor sits or lies down.
On video it is one moment. The episode restarts from frame 0 once a termination condition fires, so the robot snaps back to its initial pose.
walk3_subject1 at 195 s, anchor height.
walk3_subject4 at 218 s, ankle and wrist height.
Retargeting Matters(arXiv:2510.02252) reports 96-100 % on the same LAFAN1, the same G1 and the same BeyondMimic, and states the two steps that account for the gap: it excludes motions with complex environment interaction, and it offsets each retarget by its mean minimum body height. Neither was done here.
kobe — does it hold outside LAFAN1
All seventeen above are LAFAN1 walking and dance, with no high-difficulty one-shot motion among them. One ASAP motion, kobe, was run through the same pipeline: 206 frames, 4.1 s.
The full 4.1 s. Isaac Lab on the left, MuJoCo on the right. Over 100 MuJoCo trials: BeyondMimic success 1.000, PolySim success 0.990, global body error 128.0 mm. No transfer loss here either.
One clip is not a sample. What it supports is that transfer holds outside LAFAN1, and nothing beyond that.
This clip is not placed against PolySim's Table III, for three reasons. The 0.100 cited there for IsaacSimDR → MuJoCo is the single-simulator DR baseline the table exists to argue against, not PolySim's own result; PolySim's row is the last one, IsaacSim+IsaacGym+Genesis, at 1.000. The paper names neither its 14 nor its 5 evaluation motions — Kobe is the only motion named in the text — so the set cannot be matched. And the trainer differs: PolySim uses HumanoidVerse with ASAP rewards and teacher-student, this uses BeyondMimic. Retargeting Matters reports sim2sim success mostly at 100 % on the same LAFAN1, G1 and BeyondMimic, so the 1.000 here is the ordinary value for this lineage, not a win.
eval_polysim/<sequence>.json holds the Isaac side under the 0.5 m criterion, next to the original eval/ numbers under Isaac's own thresholds.
Source data and license
The reference motions derive from LAFAN1 by Ubisoft, released under CC BY-NC-ND 4.0. That license does not permit sharing adapted material, so the retargeted motion data is not included here — only the trained weights and the measured numbers. The onnx export carries the actor network alone and no reference motion.
Use is non-commercial. Credit Ubisoft for LAFAN1.