mlvbench-review/MedHorizon

MedHorizon

MedHorizon is a long-context medical video benchmark for evaluating multimodal models on full-procedure clinical videos. The benchmark emphasizes two properties that are not captured by short-clip medical video datasets: extremely sparse evidence retrieval and multi-hop reasoning over observations distributed across a full procedure.

Dataset Contents

• —Videos: 340 full-procedure videos.
• —Questions: 1,253 multiple-choice QA pairs.
• —Evaluation split: test.
• —Video hours: 759 hours in total.
• —Modalities and scenarios: endoscopy, surgical videos, and ultrasound studies.

The main annotation file is medhorizon_test.jsonl. Each record corresponds to one video and contains metadata plus a list of QA items under qa. The video_path field is relative to this repository, for example real_colon/002-010.mp4 or us_study/01-lus.mp4.

For compatibility with internal evaluation scripts, medhorizon_testbench.jsonl contains the same records as medhorizon_test.jsonl, and medhorizon_test_swift.jsonl provides a Swift-friendly flattened view.

Fields

• —key: video identifier.
• —dataset: source subset name.
• —organ: organ or anatomical target category.
• —scene_type: clinical scene type.
• —duration_tier: duration bucket.
• —video_path: relative path to the video file in this repository.
• —num_frames, fps, duration_seconds: video metadata.
• —qa: list of multiple-choice QA items with question, options, answer, task_id, task_name, and category metadata.
• —split: always test for the released benchmark.

Splits

This release is evaluation-only and intentionally provides a test split rather than a train split. The benchmark is intended for standardized model evaluation and leaderboard reporting.

Citation

A paper citation will be added after the preprint is available.