Philip-MIT/SOLE-R1-8B

SOLE-R1-8B

SOLE-R1-8B is a video-language reward reasoning model for robotics. It is designed to estimate task progress from robot video frames and a natural-language task description, producing both per-timestep reasoning traces and scalar progress predictions that can be used as rewards for online robot reinforcement learning.

This model accompanies the paper “SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL” by Philip Schroeder, Thomas Weng, Karl Schmeckpeper, Eric Rosen, Stephen Hart, and Ondrej Biza.

• —Paper: https://arxiv.org/abs/2603.28730
• —Project page: https://philip-mit.github.io/sole-r1/
• —Code: https://github.com/Philip-MIT/sole-r1-model
• —Training data: https://huggingface.co/datasets/Philip-MIT/soletrainingdata

Model Description

SOLE-R1 predicts robot task progress from visual observations. Given a video and a task description, the model outputs a reasoning trace and a scalar progress estimate.

Expected output format:

<think>reasoning about task progress</think><answer>progress%</answer>

The progress estimate is intended to serve as a dense reward signal for robotic reinforcement learning, especially when manually engineered rewards are unavailable.

Quick Start

The recommended interface for inference is RewardGen:

# pip install -U rewardgen

from rewardgen import generate, video_plot

# testvideos provided at the github repo: https://github.com/Philip-MIT/rewardgen videopaths = [ "testvideos/robosuite/lift/unsuccessful/robosuiteliftepisode12unsuccessfulmaxreward38.mp4" ]

task_description = "Pick up the cube from the table."

rewards, reasoningtraces = generate( model="SOLE-R1", taskdescription=taskdescription, videopaths=videopaths, viewtypepervideo=["external and wrist"], verbose=False, ) print(rewards) print(reasoning_traces)

# Plotting with showreasoningtraces=True outputsole = {"model": "SOLE-R1", "rewards": rewards[0], "reasoningtraces": reasoningtraces[0]} videoplot( outputs=[outputsole], plotsavepath='modeloutputs/sole-r1/robosuite/lift/unsuccessful/robosuiteliftepisode12unsuccessfulmaxreward38.mp4', videopath=videopaths[0], showreasoningtraces=True, taskdescription=task_description, verbose=False )

Optional pre-download:

from rewardgen.utils.modelutils import getmodel_dir

getmodeldir("sole-r1")

Input Format

The model is trained to reason over robot task progress using prompts that include:

• —A robot task description
• —The first timestep progress, typically 0%
• —The previous timestep progress
• —Visual observations from the first, previous, and current timesteps
• —Multiple camera views when available, such as external and wrist cameras

Example task description:

Pick up the cube from the table.

Output Format

The expected output format is:

<think>[reasoning about visual task progress]</think><answer>[current task progress]%</answer>

Example:

<think>The gripper has moved closer to the cube but has not yet grasped or lifted it. This indicates incremental progress from the previous timestep.</think><answer>22%</answer>

Downstream systems should parse the numeric value inside <answer>...</answer> as the reward/progress estimate.

Training Data

The model was trained on the SOLE-R1-8B training dataset.

The dataset contains robot task progress examples with images, prompts, reasoning completions, and progress labels.

It also includes a diverse collection of general spatial and multi-frame temporal reasoning data (e.g., from SSR-CoT, SpatialVLM, Spot-the-diff, Embodied CoT, RoboVQA, Robo2VLM-Reasoning) to serve as a foundational layer of our training mixture.

The full dataset is approximately 2TB.

Streaming example:

from datasets import load_dataset

ds = loaddataset( "Philip-MIT/soletraining_data", split="train", streaming=True, )

for row in ds: print(row) break

Citation

BibTeX:

@misc{schroeder2026soler1, title={SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL}, author={Philip Schroeder and Thomas Weng and Karl Schmeckpeper and Eric Rosen and Stephen Hart and Ondrej Biza}, year={2026}, eprint={2603.28730}, archivePrefix={arXiv}, primaryClass={cs.RO} }

License

This repository is released under the MIT License.