Richard1999/Event-Bench
Towards Event-oriented Long Video Understanding [📖 arXiv Paper] 👀 Overview We introduce Event-Bench, an event-oriented long video understanding benchmark built on existing datasets and human annotations. Event-Bench consists of three event understanding abilities and six event-related tasks, including 2,190 test instances to comprehensively evaluate the ability to understand video events. Event-Bench provides a systematic comparison across different… See the full description on the dataset page: https://huggingface.co/datasets/Richard1999/Event-Bench.
Towards Event-oriented Long Video Understanding
<font size=3><div align='center'> [[📖 arXiv Paper]()]</div>
👀 Overview
We introduce Event-Bench, an event-oriented long video understanding benchmark built on existing datasets and human annotations. Event-Bench consists of three event understanding abilities and six event-related tasks, including 2,190 test instances to comprehensively evaluate the ability to understand video events. <p align="center"> <img src="./asset/fig_benchmark.jpg" width="100%" height="100%"> </p>
Event-Bench provides a systematic comparison across different kinds of capabilities for existing video MLLMs, and points out the major shortcomings of open-source MLLMs.
🔍 Dataset
Download the raw videos in VNBench from the google drive link.
License:
Event-Bench is only used for academic research. Commercial use in any form is prohibited.🔮 Evaluation Pipeline
Please refer to https://github.com/RUCAIBox/Event-Bench
📈 Experimental Results
- Evaluation results of different Video MLLMs.
<p align="center"> <img src="./asset/performance.png" width="96%" height="50%"> </p>
Citation
If you find our work helpful for your research, please consider citing our work.