Datasetpublic

ShareLab-SII/VA-Judger-Bench

VA-Judger-Bench VA-Judger-Bench is a paired audio/video preference benchmark with 1,150 cases: easy: 400 cases indomain: 250 cases outdomain: 500 cases Layout Each split contains data.jsonl and a videos/ directory. All video paths are relative to the split directory. VA-Judger-Bench/ ├── README.md ├── easy/ │ ├── data.jsonl │ └── videos/ ├── indomain/ │ ├── data.jsonl │ └── videos/ └── outdomain/ ├── data.jsonl └── videos/ Record… See the full description on the dataset page: https://huggingface.co/datasets/ShareLab-SII/VA-Judger-Bench.

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes346downloads

Dataset Card

VA-Judger-Bench

VA-Judger-Bench is a paired audio/video preference benchmark with 1,150 cases:

—easy: 400 cases
—indomain: 250 cases
—outdomain: 500 cases

Layout

Each split contains data.jsonl and a videos/ directory. All video paths are relative to the split directory.

text

VA-Judger-Bench/
├── README.md
├── easy/
│   ├── data.jsonl
│   └── videos/
├── indomain/
│   ├── data.jsonl
│   └── videos/
└── outdomain/
    ├── data.jsonl
    └── videos/

Record format

json

{
  "case_id": "stable unique case identifier",
  "text_prompt": "generation prompt",
  "video_1_relative_path": "videos/example1.mp4",
  "video_2_relative_path": "videos/example2.mp4",
  "human_preference_answer": "video 1 is better",
}

human_preference_answer is either video 1 is better or video 2 is better. Use case_id, not sample_id, when joining records.

Licensing

This dataset is released under the MIT License.