d3d3shan/pi05-libero-plus-robot-initial-failures

pi0.5 LIBERO-plus Robot Initial State Failures

Failure summary artifacts for evaluating TensorAuto/tPi0.5-libero on the LIBERO-plus Robot Initial States perturbation subset through OpenTau.

Evaluation Setup

• —Benchmark: LIBERO-plus
• —Suite: libero_10
• —Perturbation category: Robot Initial States
• —Policy: TensorAuto/tPi0.5-libero
• —Tasks: 393
• —Episodes per task: 5
• —Total evaluated episodes: 2430
• —Seed schedule: episode seeds 1000 to 1004
• —Episode length: 520 steps
• —Hardware used locally: RTX 5070 Ti Laptop GPU, 12GB VRAM

Summary

• —Successes: 1738 / 2430
• —Success rate: 71.52%
• —Failures: 692
• —Failure videos rendered into grids: 504
• —Failures missing videos: 71

Files

• —videos/: 16-panel failure grid videos.
• —metadata/robot_initial_full_failure_grids.json: machine-readable failure records, task IDs, episode IDs, seeds, difficulty levels, robot initial state IDs, rewards, and source video paths.
• —metadata/robot_initial_full_failure_grids.md: human-readable legend matching failure indices to tasks and videos.
• —metadata/robot_initial_low_l1_l3_failed_rerun_manifest.json: low-difficulty failed-task rerun manifest.
• —metadata/eval_manifest.json: evaluation chunk configuration.
• —logs/robot_initial_queue.log: full local queue log.

The grid videos intentionally group failures for quick visual inspection. The detailed mapping is in metadata/robot_initial_full_failure_grids.md and metadata/robot_initial_full_failure_grids.json.