lmms-eval/VideoMMMU

This dataset contains the data for the paper Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos. Video-MMMU is a multi-modal, multi-disciplinary benchmark designed to assess LMMs' ability to acquire and utilize knowledge from videos. Project page: https://videommmu.github.io/ Leaderboard (last updated: 07 Feb, 2025) Model Overall Perception Comprehension Adaptation Δknowledge Human Expert 74.44 84.33 78.67 60.33 +33.1… See the full description on the dataset page: https://huggingface.co/datasets/lmms-eval/VideoMMMU.