lerobot/g1_mujoco

Unitree G1 · MuJoCo

A Unitree G1 standing and walking under a real lerobot locomotion policy, in your browser. The physics run headless in MuJoCo on the Space; the robot you see is drawn client-side by three.js from a stream of joint angles, so there is no video encoding and no server-side rendering.

You can shove it around and watch the policy fight to stay upright.

Using it

• —Policy buttons switch between the available controllers. Loading one takes a few seconds, during which a gantry holds the robot up — the badges tell you when that is happening.
• —Drag the robot to shove it. Drag empty space to orbit, scroll to zoom, c toggles the chase camera.
• —WASD to walk, QE to turn, or use the sliders.
• —GR00T adds a squat height slider, with R/F to ramp it.
• —SONIC is whole-body, so instead of a joystick it exposes the 64 latent motion-token sliders its decoder consumes.
• —Joints the active policy leaves alone become directly posable, which is why an arms panel appears for the lower-body-only policies.

How it fits together

Two processes, so the controller can crash, be swapped, or be restarted without taking physics down with it. They talk over DDS exactly as they would on a real G1, which is the point: the controller is a plain DDS peer publishing rt/lowcmd and subscribing to rt/lowstate, so it neither knows nor cares that it is driving a simulator.

app/ three.js viewer: plain ES modules, an import map, and a vendored three.js, so there is no build step sim/g1simweb.py physics + pose stream + static file server (:7860) sim/g1controller.py DDS peer that ticks the selected lerobot policy sim/controllerextras.py the bits the viewer needs that lerobot does not advertise (see below) tools/ regenerate the viewer's assets from the MuJoCo model

lerobot is installed from upstream main and is not patched. Two things the UI needs aren't expressible through lerobot's RobotController protocol, so they are adapted from the outside in sim/controller_extras.py:

• —Controller discovery. lerobot's make_robot_controller matches a hardcoded list, so a newly added controller would be offered and then rejected. Here the controllers/ package is scanned instead, by parsing the modules rather than importing them — importing pulls in onnxruntime and every policy's dependencies just to answer "what can I pick?", and would let one controller with an unmet optional dependency hide all the others. Dropping a module into controllers/ is genuinely enough to make it selectable.
• —Scalar commands. Things a policy takes as an observation and simply holds, such as GR00T's commanded base height, are only reachable as an attribute. A small table maps a UI key onto that attribute.

Local development

docker build -t g1-demo .
docker run --rm -p 7860:7860 g1-demo   # then open localhost:7860

The image installs lerobot from main at build time, and the build re-checks the ref so a rebuild really does pick up new upstream commits. To point at something else, --build-arg LEROBOT_REF=<branch-or-tag>; to test local library changes without a rebuild, bind-mount a checkout over it:

docker run --rm -p 7860:7860 -v "$PWD/../src:/src" -e LEROBOT_SRC=/src g1-demo

Regenerating the viewer's assets (only needed if the robot model changes):

cd tools
./mjcf_to_kinematics.py --mjcf <g1_29dof.xml> --out ../app/public/robot/g1/kinematics.json
./pack_meshes.py --kinematics ../app/public/robot/g1/kinematics.json --out ../app/public/robot/g1/g1.glb
./verify_fk.py --mjcf <g1_29dof.xml> --kinematics ../app/public/robot/g1/kinematics.json

verify_fk.py rebuilds the viewer's transform chain in Python and compares every body against MuJoCo's own forward kinematics over random poses, which is what catches a bad joint mapping or a stale export.