rewardbench
reward-bench-results
Results for Holisitic Evaluation of Reward Models (HERM) Benchmark
Here, you'll find the raw scores for the HERM project.
The repository is structured as follows.
├── best-of-n/ <- Nested directory for different completions on Best of N challenge
| ├── alpaca_eval/ └── results for each reward model
| | ├── tulu-13b/{org}/{model}.json
| | └── zephyr-7b/{org}/{model}.json
| └── mt_bench/
|… See the full description on the dataset page: https://huggingface.co/datasets/allenai/reward-bench-results.reward-bench
Code | Leaderboard | Prior Preference Sets | Results | Paper
Reward Bench Evaluation Dataset Card
The RewardBench evaluation dataset evaluates capabilities of reward models over the following categories:
Chat: Includes the easy chat subsets (alpacaeval-easy, alpacaeval-length, alpacaeval-hard, mt-bench-easy, mt-bench-medium)
Chat Hard: Includes the hard chat subsets (mt-bench-hard, llmbar-natural, llmbar-adver-neighbor, llmbar-adver-GPTInst, llmbar-adver-GPTOut… See the full description on the dataset page: https://huggingface.co/datasets/allenai/reward-bench.reward-bench-2Code | Leaderboard | Results | Paper
RewardBench 2 Evaluation Dataset Card
The RewardBench 2 evaluation dataset is the new version of RewardBench that is based on unseen human data and designed to be substantially more difficult! RewardBench 2 evaluates capabilities of reward models over the following categories:
Factuality (NEW!): Tests the ability of RMs to detect hallucinations and other basic errors in completions.
Precise Instruction Following (NEW!): Tests the ability of RMs… See the full description on the dataset page: https://huggingface.co/datasets/allenai/reward-bench-2.reward-bench-2-resultsrewardbenchVC-RewardBench
Visual-ERM
Visual-ERM is a multimodal generative reward model for vision-to-code tasks.It evaluates outputs directly in the rendered visual space and produces fine-grained, interpretable, and task-agnostic discrepancy feedback for structured visual reconstruction.
📄 Paper |
💻 GitHub |
📊 VC-RewardBench
Model Overview
Existing rewards for vision-to-code usually fall into two categories:
Text-based rewards such as edit distance or TEDS, which ignore… See the full description on the dataset page: https://huggingface.co/datasets/internlm/VC-RewardBench.