Datasetpublic

rahul-ai-01/groot_n1.7_inference_on_diff_data

GROOT Inference Analysis Log Evaluation records for a GR00T policy trained on the task "pick octopus and place inside brown basket", run on a Unitree G1 at 20 Hz with the ego_view stereo camera. Six training checkpoints (50, 100, 150, 200, 250, 300 demonstration episodes) were each evaluated on 50 inference episodes. Every episode is recorded here with video, per-tick state/action logs and run metadata. Success rate Checkpoint (training episodes) Success… See the full description on the dataset page: https://huggingface.co/datasets/rahul-ai-01/groot_n1.7_inference_on_diff_data.

sourceHugging Faceupdated 26d agoView on Hugging Face

0likes7kdownloads

Dataset Card

GROOT Inference Analysis Log

Evaluation records for a GR00T policy trained on the task "pick octopus and place inside brown basket", run on a Unitree G1 at 20 Hz with the ego_view stereo camera.

Six training checkpoints (50, 100, 150, 200, 250, 300 demonstration episodes) were each evaluated on 50 inference episodes. Every episode is recorded here with video, per-tick state/action logs and run metadata.

Success rate

[image]

Checkpoint (training episodes)	Success	Success rate	Failed episodes
50	47 / 50	94 %	03, 32, 36
100	48 / 50	96 %	08, 19
150	50 / 50	100 %	—
200	48 / 50	96 %	05, 08
250	49 / 50	98 %	14
300	50 / 50	100 %	—

Overall: 292 / 300 successful episodes (97.3 %).

What the numbers say

—Success rate climbs from 94 % → 100 % between the 50- and 300-episode checkpoints; the last checkpoint clears all 50 episodes.
—The curve is not monotonic: 150 already reached 100 % and 200 fell back to 96 %. With 50 episodes per checkpoint, a 1-episode difference is 2 percentage points (95 % CI on 48/50 is roughly 86–99 %), so 96 %, 98 % and 100 % here are not statistically separable. Treat 150–300 as "converged, ~98 % ± 2".
—Episode duration is the clearer improvement signal — the policy gets faster, not just more reliable:

Checkpoint	Mean duration	Median duration
50	29.4 s	25.6 s
100	27.7 s	24.6 s
150	29.1 s	28.3 s
200	24.7 s	20.8 s
250	20.4 s	18.7 s
300	20.4 s	20.4 s

(computed from duration_s in every episode's meta.json)

Repository layout

50_inference_recordings/          checkpoint trained on 50 episodes
100_inference_recordings/
150_inference_recordings/
200_inference_recordings/
250_inference_recordings/
300_inference_recordings/
    inference_01/ ... inference_50/     one directory per evaluation episode
        meta.json     run config, timing, tick/frame counts, status
        steps.csv     per-tick robot state, policy actions, published commands
        chunks.csv    per-inference-chunk records
        chunks/       raw action chunks
        video.mp4     ego_view recording (20 fps, 640x480)
    notes.txt         hand-scored pass/fail for that checkpoint (source of truth)

VLADataLens/          viewer / analysis tool for the recordings
plot_success_rate.py  regenerates success_rate.png from notes.txt data
success_rate.png      the chart above

Notes:

—150_inference_recordings/ contains 52 episode directories; only the first 50 were scored (inference_51, inference_52 are extra runs).
—Each checkpoint directory is ~250–380 MB, mostly video.

Reproducing the chart

The pass/fail data lives in the RESULTS dict at the top of plot_success_rate.py and mirrors the notes.txt files.

bash

./venv/bin/python plot_success_rate.py

Prints the per-checkpoint table and writes success_rate.png.

Scoring convention

An episode counts as a failure if the octopus does not end up inside the brown basket. The failing episode ids per checkpoint are listed in the table above and in each notes.txt; every other episode in that directory is a success.