witcheer/microduck-skill-tree

Microduck skill tree, the training log

Simulation only. Nothing here has run on a real robot yet; the Microduck this is for is still on order. Every policy, curve and clip in this repo comes from mjlab / MuJoCo Warp on one RTX 5090, trained with the tasks in pollen-robotics/microduck_rl at commit 53b8971.

This is the build log. One folder per level of the skill tree, publishable or not, partial or not. The installable policies live in their own model repos (one per policy, in the shape robotctl installs); this dataset holds everything around them: the reward curve of every run, the exact queue item that ran it, the proof-take logs the verdict was read from, the clips, and the manifests.

Sibling dataset: witcheer/microduck-hermes-runs logs every run where Hermes Agent drives the simulated duck from one typed sentence, with the agent's full transcript, the simulator's ground truth and a replay video. Both datasets and every policy repo are grouped in the Microduck skill tree collection.

The ladder

All budgets are the maker's max_iterations from the task config unless stated. 4096 parallel environments, 24 steps per iteration, PPO from rsl_rl, tensorboard logging.

What is in a level folder

• —reward-curve.csv: every scalar rslrl logged per iteration (`Train/meanreward, Train/meanepisodelength, Perf/totalfps`, the `EpisodeTermination/*` terms), pulled from the tfevents of every run of that level and concatenated by iteration.
• —reward-curve.png: mean reward per iteration, raw and a 50-iteration moving average.
• —queue-item.sh: the exact header that trained it. The shared body it sources is tools/duck-train-body.sh (resumable, checkpoint every 100 iterations, exports with the maker's scripts/export.py).
• —manifest.json: the schema-2 manifest uv run publish wrote for the policy repo, when there is one.
• —proof-takes/*.log: the recorder's per-step trunk height and body-frame gravity for each take the verdict was read from (levels 5, 6 and 7 so far; level 7 adds the ball position on every line). Verdicts are read from these numbers, not from the video.
• —clip-*.mp4: iteration 0 against the trained policy, same camera, same spawn, for levels 1, 5 and 6; the trained policy alone for levels 3 and 7 (level 7 holds the opening frame for about a second before the take plays, and the camera follows the ball). Level 2 has no clip because the first one I made was wrong (an environment reset at the 6 s episode boundary read as a stand-up) and a correct one is not recorded yet.

Tools

• —tools/headless_play.py: records any checkpoint of any mjlab task over ssh with no display. Both of mjlab's play viewers fail headless (viser asserts on a degenerate command slider, the native one wants X11); this wrapper swaps them for a plain policy loop, lets play's own recorder write the mp4, forces the spawn state, overrides the episode length, prints every reset and samples trunk height and body-frame gravity at a chosen cadence. Copy it into the microduck_rl project dir and run with uv run python.
• —tools/duck-train-body.sh and tools/duck-lane.sh: the queue item body and the weekday duck/bench alternation. Written for one specific box (paths under /home/witcheer); read them for the shape, not to run as-is.

Things learnt on the way, in order

• —A queue item that adds a rendering flag needs a real probe of the exact argv, not a help parse: --video swallowed the next token (tyro wants --video True), and the offscreen renderer needs MUJOCO_GL=egl set by the item itself. Two training blocks lost.
• —A resumed rsl_rl run counts the loaded iteration as its first, so --agent.max-iterations TARGET-IT stops one short of TARGET forever. Use TARGET-IT+1.
• —A recording longer than the task's episode contains a reset that teleports the duck to a fresh spawn and reads as the skill happening. Know episode_length_s before recording; override it for the clip; print every reset.
• —Timed terminations do the same thing periodically: VelStand resets a duck that has been down for 8 s, so every "get-up" landing exactly 8 s after a fall was a reset. Read terminations[ in the task config before reading any recovery off the numbers.
• —Read verdicts from trunk height and body-frame gravity, not from frames. Vision reads of small contact sheets were wrong three times in one afternoon.
• —A one-second skill needs 0.1 s sampling; at 0.5 s the roll showed as upright, upright, nothing in between.
• —The roll task deletes the fall termination (a roll starts with a fall) and spawns half its episodes mid-roll with the chin tucked, so the hard second half is practised from iteration 1. Its reward was flat from iteration 2,000 of 10,000.
• —The kick task gives the policy no ball observation at all, because the real robot has no ball sensor; only the critic sees the ball. The ball is placed at the right toe at every reset and the policy kicks on timing. Its reward reached 92.9 by iteration 1,000 and ended at 90.1 at 10,000. 14 of its 10,001 logged iterations have a negative mean reward (the lowest -4,686.8 at iteration 4,361) while the mean episode length stays at about 246 to 250 steps, so these are not mass falls; the cause is not investigated yet. The chart clips the 8 deepest and says so; every raw value is in the csv.
• —A moving target needs a camera that follows it. A trunk-tracking camera lost the ball by 1.5 s, a fixed camera lost it out of a different edge each take (the spawn heading is random), and a wide shot shrank the duck to a speck. Tracking the ball at 0.9 m keeps both in frame for the first second or so; the numbers cover the rest.

Posts

Each level was posted the day it landed: level 1, level 3, level 5, level 6, level 7.

The real robot arrives in a few months. Until it does, every number in this repo is a simulation number.