spa-bench
Spa_Bench_Full_MolmoAct2
Spa-Bench physical rollouts — MolmoAct2
Physical SO-101 rollout recordings for the MolmoAct2 Spa-Bench checkpoint at
epoch 12, step 76,596.
Scope and counts
This is a complete Spa-Bench evaluation: 900 episodes, comprising 300
primary withheld/OOD spatial trials and 600 diagnostic/control trials. The
repository contains 782,590 frames, 356 unique instruction strings, two
480×640 RGB streams, and six-dimensional state/action at 30 FPS.
Reported physical… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Full_MolmoAct2.Spa_Bench_Full_Pi0.5
Spa-Bench physical rollouts — pi0.5
Physical SO-101 rollout recordings for the pi0.5 Spa-Bench checkpoint at epoch
12, step 76,596.
Scope and counts
The scored benchmark protocol is episodes 0–899: 300 primary withheld/OOD
spatial trials and 600 diagnostic/control trials. Their prompt sequence matches
the MolmoAct2 and GR00T-N1.7 Frozen LLM full releases exactly.
This repository contains 920 episodes, 781,538 frames, 356 unique instruction
strings, two 480×640 RGB… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Full_Pi0.5.Spa_Bench_Partial_GR00T-N1.7_Full_Fine-Tune
Spa-Bench physical rollouts — GR00T-N1.7 Full Fine-Tune (partial)
Physical SO-101 rollout recordings for the epoch-12 GR00T-N1.7 Full
Fine-Tune Spa-Bench checkpoint.
Scope and counts
This is a partial familiar/in-distribution baseline, not a complete Spa-Bench
evaluation. It contains 120 episodes—20 from each of the six task
families—plus 127,143 frames, 90 unique instruction strings, two 480×640 RGB
streams, and six-dimensional state/action at 30 FPS. It contains… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Partial_GR00T-N1.7_Full_Fine-Tune.Spa_Bench_Full_GR00T-N1.7_Frozen_LLM
Spa-Bench physical rollouts — GR00T-N1.7 Frozen LLM
Physical SO-101 rollout recordings for the epoch-12 GR00T-N1.7 Frozen LLM
Spa-Bench checkpoint. The LLM parameters were frozen; visual, multimodal, and
diffusion-action components were adapted.
Scope and counts
This is a complete Spa-Bench evaluation: 900 episodes, comprising 300
primary withheld/OOD spatial trials and 600 diagnostic/control trials. The
repository contains 830,614 frames, 356 unique instruction… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Full_GR00T-N1.7_Frozen_LLM.Spa_Bench_Partial_VLA-0
Spa-Bench physical rollouts — VLA-0 (partial)
Physical SO-101 rollout recordings for the VLA-0 Spa-Bench checkpoint at the
end of epoch 12.
Scope and counts
This is a partial familiar/in-distribution baseline, not a complete Spa-Bench
evaluation. It contains 72 episodes, with at least ten from each of the six
task families, plus 11,051 frames, 51 unique instruction strings, two 480×640
RGB streams, and six-dimensional state/action at 10 FPS. It contains no OOD
or… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Partial_VLA-0.spa-bench-eval-rollouts-pi-0-5-full
Spa-Bench Evaluation Rollouts — pi0.5 (Full)
Open this dataset in the LeRobot visualizer
This repository contains physical Spa-Bench rollout trajectories with synchronized middle and wrist RGB video, robot state, action, timestamps, episode indices, and task indices. The native Hugging Face Data Studio viewer is enabled through the Parquet files declared above.
Dataset summary
Coverage: 920 recordings; episodes 0–899 form the scored protocol and 900–919 are… See the full description on the dataset page: https://huggingface.co/datasets/Spa-Bench/spa-bench-eval-rollouts-pi-0-5-full.