SpatialReward/SpatialReward-8B

1---2license: apache-2.03base_model: Qwen/Qwen3-VL-8B-Instruct4tags:5  - reward-model6  - image-editing7  - reinforcement-learning8  - spatial-reasoning9  - vision-language-model10  - icml202611datasets:12  - SpatialReward/SpatialReward-Train13pipeline_tag: image-text-to-text14language:15  - en16---17 18<p align="center">19  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/logo.png" width="65%">20</p>21 22<p align="center">23  <a href="https://lorangan-ddup.github.io/SpatialReward/"><img src="https://img.shields.io/badge/Project%20Page-SpatialReward-yellow" alt="project page"></a>24  <a href="https://arxiv.org/abs/2602.07458"><img src="https://img.shields.io/badge/arXiv-2602.07458-b31b1b.svg" alt="arxiv"></a>25  <a href="https://github.com/lorangan-ddup/SpatialReward"><img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="github"></a>26  <a href="https://huggingface.co/datasets/SpatialReward/MER-Bench"><img src="https://img.shields.io/badge/MER--Bench-🤗-yellow" alt="dataset"></a>27  <a href="https://huggingface.co/datasets/SpatialReward/SpatialReward-Train"><img src="https://img.shields.io/badge/Training--Data-🤗-yellow" alt="dataset"></a>28</p>29 30<h4 align="center">31    <p>32        <a href=#-news>News</a> |33        <a href=#-introduction>Introduction</a> |34        <a href=#-quick-start>Quick Start</a> |35        <a href=#-benchmark-evaluation>Benchmark Evaluation</a> |36        <a href=#️-citing-us>Citation</a>37    </p>38</h4>39 40**SpatialReward** is a state-of-the-art reward model for instruction-guided image editing that addresses the critical "Attention Collapse" problem through explicit spatial reasoning. By anchoring semantic judgments to predicted edit regions via bounding boxes, SpatialReward achieves unprecedented accuracy and reliability as both an evaluator and RL training signal.41 42<p align="center">43  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/attention_visualization.png" width="95%">44  <br>45  <em>Visualizing the Attention Collapse problem vs. SpatialReward's spatial grounding.</em>46</p>47 48## 🔥 News49 50- **2026-05-05**: 🎉 We have open-sourced the **SpatialReward-8B** model weights, **[MER-Bench](https://huggingface.co/datasets/SpatialReward/MER-Bench)** benchmark, and **[SpatialReward-Train](https://huggingface.co/datasets/SpatialReward/SpatialReward-Train)** (260k spatial-aware training data)!51- **2026-05-01**: 🎉 **SpatialReward** has been accepted to **ICML 2026**!52- **2026-02-12**: We have released the **inference code**, **reward server**, and **training configurations**!53- **2026-02-07**: The paper is available on [arXiv](https://arxiv.org/abs/2602.07458).54 55## 📌 Introduction56 57Online Reinforcement Learning (RL) holds immense potential for advancing instruction-guided image editing, but its progress has been severely hindered by a critical perception gap we term **"Attention Collapse"**. Existing reward models frequently neglect cross-image comparisons and fail to capture fine-grained editing details, leading to inaccurate evaluations and unstable RL training.58 59To overcome this, we propose **SpatialReward**, which:60- **Introduces MER-Bench**: A new benchmark featuring multi-edit scenarios and expert human annotations for measuring reward model quality.61- **Enforces spatial reasoning**: Predicts bounding boxes for edit regions and anchors semantic judgments to pixel-level evidence.62 63<p align="center">64  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/performance_table.png" width="95%">65  <br>66  <em>Comprehensive benchmark results. SpatialReward achieves SOTA performance, outperforming GPT-4.1 and GPT-5 on MER-Bench.</em>67</p>68 69<p align="center">70  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/merbench_category_breakdown.png" width="70%">71  <br>72  <em>MER-Bench performance breakdown by editing category.</em>73</p>74 75## 🚀 Quick Start76 77### Installation78 79```bash80git clone https://github.com/Kwai-Keye/SpatialReward.git81cd SpatialReward82 83conda create -n spatialreward python=3.11 -y84conda activate spatialreward85 86pip install torch==2.8.0 torchvision --extra-index-url https://download.pytorch.org/whl/cu12687pip install -r requirements.txt88```89 90### Reward Server91 92```bash93# Start reward server94cd example/reward/server95bash start_servers.sh96bash start_proxy.sh97 98# Query from client99from example.reward.client.reward_client_edit import RewardClient100 101client = RewardClient(proxy_host="127.0.0.1", proxy_port=23456)102scores, rewards, reasoning, meta_data = client.evaluate(103    input_images=[input_img],104    output_image=[output_img],105    meta_datas=[{"instruction": "Remove the dog"}]106)107```108 109## 📊 Benchmark Evaluation110 111Model and data are loaded directly from HuggingFace by default.112 113```bash114# MER-Bench115bash eval/MERBench/run.sh116 117# MMRB2118bash eval/MMRB2/run.sh119 120# EditReward-Bench121bash eval/EditReward-Bench/run.sh122```123 124## 📚 Datasets125 126| Dataset | Description | Link |127|---|---|---|128| **SpatialReward-Train** | 260k spatial-aware training data (SFT + RL) | [🤗 Hub](https://huggingface.co/datasets/SpatialReward/SpatialReward-Train) |129| **MER-Bench** | MultiEditReward-Bench evaluation benchmark | [🤗 Hub](https://huggingface.co/datasets/SpatialReward/MER-Bench) |130 131## 🎯 Training132 133### SFT (LLaMA-Factory)134```bash135llamafactory-cli train example/SpatialReward-train/sft/qwen3vl_lora_spatial_reward.yaml136```137 138### RL (ms-swift / GRPO)139 140```bash141# Replace ORM first142cp example/SpatialReward-train/rl/orm.py <ms-swift>/swift/plugin/orm.py143bash example/SpatialReward-train/rl/run_mater.sh144```145 146### RL Results on OmniGen2147 148<p align="center">149  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/omnigen2_rl_results.png" width="85%">150  <br>151  <em>SpatialReward delivers +0.90 on GEdit-EN Overall, doubling GPT-4.1's gain (+0.45).</em>152</p>153 154<p align="center">155  <img src="https://huggingface.co/SpatialReward/SpatialReward-8B/resolve/main/assets/rl_training_curves.png" width="95%">156  <br>157  <em>Stable RL training dynamics with SpatialReward as reward signal.</em>158</p>159 160## 🙏 Acknowledgements161 162We thank [EditScore](https://github.com/VectorSpaceLab/EditScore) and [EditReward](https://github.com/TIGER-AI-Lab/EditReward) for valuable references.163 164## ❤️ Citing Us165 166```bibtex167@article{long2026spatialreward,168  title={SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning},169  author={Long, Yancheng and Yang, Yankai and Wei, Hongyang and Chen, Wei and Zhang, Tianke and Fan, Haonan and Liu, Changyi and Jiang, Kaiyu and Chen, Jiankang and Tang, Kaiyu and Wen, Bin and Yang, Fan and Gao, Tingting and Li, Han and Yang, Shuo},170  journal={arXiv preprint arXiv:2602.07458},171  year={2026}172}173```174 175## 📄 License176 177Apache 2.0178