Auryal/libero-microwave-grm-rollouts
LIBERO Microwave GRM Rollouts Dense-reward-annotated rollout dataset from GRPO training of OpenVLA-OFT on LIBERO-10 Task 9 ("put the yellow and white mug in the microwave and close it"). Dataset Stat Value Episodes ~2,100 (T >= 5 steps) Format LeRobot (parquet + images) Task put the yellow and white mug in the microwave and close it Size 41 GB Reward model Robo-Dopamine GRM-3B Policy OpenVLA-OFT (LoRA, GRPO-trained) Simulator LIBERO… See the full description on the dataset page: https://huggingface.co/datasets/Auryal/libero-microwave-grm-rollouts.
LIBERO Microwave GRM Rollouts
Dense-reward-annotated rollout dataset from GRPO training of OpenVLA-OFT on LIBERO-10 Task 9 ("put the yellow and white mug in the microwave and close it").
Dataset
Columns
Each parquet episode contains:
Reward Visualizations
The dataset includes ~2,088 per-episode MP4 videos (reward_vis_*.mp4) showing:
- Top row: 3 camera panels (Front, Wrist L, Wrist R)
- Bottom row: Hop reward plot (symlog) + accumulated progress plot (linear)
- Task name and step counter overlaid
Training Context
This data was collected during a 20-epoch GRPO training run that exhibited reward hacking — success rate dropped from 86.5% to 0% while policy loss converged. The dataset captures the full trajectory of policy collapse, making it useful for studying:
- Dense reward model failure modes
- Reward hacking in embodied RL
- GRM scoring patterns across policy quality levels
Early episodes (epoch 1-5) show competent manipulation. Late episodes (epoch 15+) show degenerate behavior that still receives positive GRM scores.
Associated Model
Auryal/openvla-oft-libero-microwave-grm-grpo
Citation
@misc{auryal2026grm-grpo,
title={Dense Reward RL with GRM for Robotic Manipulation},
author={Auryal},
year={2026},
}