simplecloud/VidChain-exercise

1<p align="center">2  <h1 align="center"> ✏️ Data for VidChain Excercise</h1>3  <h2 align="center">VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning</h2>4  5<p align="center">Ji Soo Lee*, Jongha Kim*, Jeehye Na, Jinyoung Park, Hyunwoo J. Kim†.6  </p>7 8  <h2 align="center">9    AAAI 2025 10  </h2>11 12  <h3 align="center">13    <a href="https://arxiv.org/pdf/2501.06761" target='_blank'><img src="https://img.shields.io/badge/arXiv-2501.06761-b31b1b.svg"></a>14    <a href="https://huggingface.co/datasets/simplecloud/VidChain-Data"><img src="https://img.shields.io/badge/huggingface-datasets-yellow"></a>15  </h3>16 17<div align="center">18  <img src="asset/main.png" width="750px" />19</div>20 21 22## 🎯 Learning Objectives23By working through this exercise, you will:24- Reproduce baseline behavior of a video-language model (**VTimeLLM**, CVPR 2024 Highlight).  25- Observe the limitations of existing approaches in temporal reasoning and coherence.  26- Implement and experiment with **VidChain's improvements** using M-DPO.  27- Run inference on videos to generate **dense temporal captions (Dense Video Captioning)**.  28- Evaluate how preference alignment improves performance over baselines.  29- Discuss potential strategies for ensembling different reasoning paths of VidChain's CoTasks.  30 31<br>32 33## Citations 🌱34```35@inproceedings{lee2025vidchain,36  title={VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning},37  author={Lee, Ji Soo and Kim, Jongha and Na, Jeehye and Park, Jinyoung and Kim, Hyunwoo J},38  booktitle={AAAI},39  year={2025}40}41```42