CodeGoat24/WorldReward-Bench
WorldReward-Bench A human-annotated preference benchmark for camera-conditioned world models. 760 pairs of videos, each pair generated by two different models from the same source image and the same camera-action sequence, with human verdicts on three independent axes. 📰 Paper: https://arxiv.org/abs/2609.03952 🪐 Project Page: https://codegoat24.github.io/WorldReward 🤗 Model Collections: https://huggingface.co/CodeGoat24/WorldReward-9B 🚀 Github:… See the full description on the dataset page: https://huggingface.co/datasets/CodeGoat24/WorldReward-Bench.
WorldReward-Bench
A human-annotated preference benchmark for camera-conditioned world models. 760 pairs of videos, each pair generated by two different models from the same source image and the same camera-action sequence, with human verdicts on three independent axes.
- 📰 Paper: https://arxiv.org/abs/2609.03952
- 🪐 Project Page: https://codegoat24.github.io/WorldReward
- 🤗 Model Collections: https://huggingface.co/CodeGoat24/WorldReward-9B
- 🚀 Github: https://github.com/CodeGoat24/WorldReward
- 👋 Point of Contact: Yibin Wang
Evaluation Dimensions
Each verdict is left, right, or tie.
Contents
bench.jsonl # 760 pairs, one JSON object per line
videos/<pair_id>/source.* # shared source image (exact path is in bench.jsonl)
videos/<pair_id>/left.mp4 # the video shown on the left
videos/<pair_id>/right.mp4 # the video shown on the right
videos/<pair_id>/left_overlay.mp4 # left.mp4 with the commanded action burned in
videos/<pair_id>/right_overlay.mp4 # right.mp4 with the commanded action burned inThe *_overlay.mp4 files are a visualisation aid for inspecting trajectories by eye.
Source images keep their original format and pixels when they are at most 2048px on the long side. Larger ones are downscaled to 2048px and saved as JPEG.
Schema
Results
Three-way agreement with the human labels (%). All 760 pairs count; a pair whose label is tie is correct only if the model also predicts tie. Act./App./Mot. = action / appearance / motion. Best and second-best per column; -- marks an axis a predictor does not model.
Usage
See https://github.com/CodeGoat24/WorldReward for the full evaluation protocol.
Citation
@article{wang2026worldreward,
title={WorldReward: Reward Modeling for Camera-Conditioned World Models},
author={Wang, Yibin and Wang, Zehan and Tang, Junshu and Li, Zhimin and Zhou, Yujie and Bu, Jiazi and Ling, Pengyang and Han, Feng and Zhang, Zhixiong and Xing, Long and others},
journal={arXiv preprint arXiv:2609.03952},
year={2026}
}