CoolFace
Modelpublic

yifanzhang114/R1-Reward

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
7likes31downloads
Model Card

<p align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/623d8ca4c29adf5ef6175615/q3Anm7o-MoNYjB8JztGVT.png" width="60%" /> </p>

<font size=3><div align='center' > [πŸ“– arXiv Paper] [πŸ“Š R1-Reward Code] [πŸ“ R1-Reward Data] </div></font>

Training Multimodal Reward Model Through Stable Reinforcement Learning

πŸ”₯ We are proud to open-source R1-Reward, a comprehensive project for improve reward modeling through reinforcement learning. This release includes:

  • β€”R1-Reward Model: A state-of-the-art (SOTA) multimodal reward model demonstrating substantial gains (Voting@15):
  • β€”13.5% improvement on VL Reward-Bench.
  • β€”3.5% improvement on MM-RLHF Reward-Bench.
  • β€”14.6% improvement on Multimodal Reward Bench.
  • β€”StableReinforce Algorithm: A novel reinforcement learning method that enhances the Reinforce++ approach by improving training loss stability, advantage estimation, and reward function design.
  • β€”Open-Source Resources: We provide the R1-Reward model, the R1-Reward RL training dataset, and inference code for IXC-Reward，MM-RLHF Reward and R1-Reward on the three benchmarks in Figure 1.

image/png

Citation

If you find it useful for your research and applications, please cite related papers/blogs using this BibTeX:

bibtex
@article{zhang2025r1,
  title={R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning},
  author={Zhang, Yi-Fan and Lu, Xingyu and Hu, Xiao and Fu, Chaoyou and Wen, Bin and Zhang, Tianke and Liu, Changyi and Jiang, Kaiyu and Chen, Kaibing and Tang, Kaiyu and others},
  journal={arXiv preprint arXiv:2505.02835},
  year={2025}
}

Related Projects