MCG-NJU/VideoChat3-OL617k

VideoChat3-OL617K

VideoChat3-OL617K is the online video instruction data used by VideoChat3. It is designed to train proactive streaming video assistants that continuously observe incoming video, accumulate visual evidence, and respond at the appropriate moment.

The dataset converts video-question-answer triples into causal streaming supervision. Visual clue intervals are first localized and verified, then transformed into streaming sequences with explicit response-state tokens: </Silence>, </Standby>, and </Response>. These tokens teach the model when to remain silent, continue collecting evidence, and provide an answer.

This repository provides JSONL annotation files. The original videos are not duplicated here; users should resolve videos from the original dataset paths listed below.

📄 Paper · 🌐 Homepage · 💻 GitHub · 🤗 Paper Page

<p align="center"> <img src="ol617k.jpg" alt="VideoChat3-LV116K Overview" width="100%"> </p>

Data Sources

VideoChat3-OL617K use the following video datasets:

The ol617k.json file provides the mapping between dataset annotations and video sources. You can refer to it when organizing the dataset structure.

Citation

If you use this data, please cite VideoChat3 and the original video datasets used by the annotations.

@misc{videochat3,
      title={VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding}, 
      author={Xinhao Li and Yuhan Zhu and Xiangyu Zeng and Yuhao Dong and Haoning Wu and Zhiqiu Zhang and Yuandong Yang and Changlian Ma and Qingyu Zhang and Yansong Shi and Xinyu Chen and Haoran Chen and Zizheng Huang and Jun Zhang and Kun Ouyang and Lin Sui and Ziang Yan and Yicheng Xu and Chenting Wang and Yinan He and Hongjie Zhang and Yi Wang and Yu Qiao and Yali Wang and Ziwei Liu and Kai Chen and Limin Wang},
      year={2026},
      eprint={2607.14935},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.14935}, 
}