kumuji/Sa2VA-i-8B

Sa2VA-i: Improving Sa2VA Results with Consistent Training and Inference

 

3rd Place Report of LSVOS 2025 MeViS Track

[Alexey Nekrasov](https://scholar.google.com/citations?user=xJW2v3cAAAAJ)¹ · [Ali Athar](https://scholar.google.com/citations?user=mexenQMAAAAJ) · [Daan de Geus](https://scholar.google.com/citations?user=4gX3HRoAAAAJ)² · [Alexander Hermans](https://scholar.google.com/citations?user=V0iMeYsAAAAJ)¹ · [Bastian Leibe](https://scholar.google.com/citations?user=ZcULDB0AAAAJ)¹

¹RWTH Aachen University · ²Eindhoven University of Technology

Abstract

Sa2VA is a recent model for language-guided dense grounding in images and video that achieves state-of-the-art results on multiple segmentation benchmarks and that has become widely popular. However, we found that Sa2VA does not perform according to its full potential for referring video object segmentation tasks. We identify inconsistencies between training and inference procedures as the key factor holding it back. To mitigate this issue, we propose an improved version of Sa2VA, Sa2VA-i, that rectifies these issues and improves the results. In fact, Sa2VA-i sets a new state of the art for multiple video benchmarks and achieves improvements of up to +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS and +4.1 on ReVOS using the same Sa2VA checkpoints. With our fixes, the Sa2VA-i-1B model even performs on par with the original Sa2VA-26B model on the MeViS benchmark. We hope that this work will show the importance of seemingly trivial implementation details and that it will provide valuable insights for the referring video segmentation field. We provide the code and updated models at this https URL

🚀 Overview

Sa2VA-i is an improved version of the popular Sa2VA model for language-guided dense grounding in images and video. While Sa2VA achieves state-of-the-art results on multiple segmentation benchmarks, we identified critical inconsistencies between training and inference procedures that limited its full potential for referring video object segmentation tasks.

Key improvements in Sa2VA-i:

• —✅ Consistent training and inference - eliminates incompatibility between finetuned mask decoder and frozen memory components of SAM2
• —✅ Improved frame sampling - uniform sampling instead of first-frame sampling
• —✅ Better mask propagation - uses original SAM2 weights for propagation while keeping finetuned decoder for initial predictions
• —✅ Significant performance gains - up to +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS

📊 Performance Highlights

Note: Sa2VA-i-1B performs on par with original Sa2VA-26B on MeViS benchmark!

🏆 Competition Results

3rd Place in LSVOS 2025 MeViS Track (RVOS) with 64.1 J&F

🤗 Model Zoo

Sa2VA-i provides improved inference procedures for existing Sa2VA models. Available models:

🎯 Quick Start

For installation and basic usage, please refer to the original Sa2VA repository. Sa2VA-i is a drop-in replacement for inference.

🔧 Key Improvements

1. Consistent Training-Inference

Eliminates incompatibility between finetuned mask decoder and frozen memory components by using the same procedure during both training and inference.

2. Improved Frame Sampling

Replaces first-frame sampling with uniform sampling for better coverage of video content.

3. Original SAM2 Mask Propagation

Uses original SAM2 weights for propagation while keeping finetuned decoder for initial mask predictions.

📚 Citation

If you use Sa2VA-i in your research, please cite:

@article{sa2va2025improved,
  title={Sa2VA-i: Improving Sa2VA Results with Consistent Training and Inference},
  author={Nekrasov, Alexey and Athar, Ali and de Geus, Daan and Hermans, Alexander and Leibe, Bastian},
  journal={arXiv preprint arXiv:2509.19082},
  year={2025}
}

Shout-out to the original Sa2VA paper!

@article{yuan2025sa2va,
  title={Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos},
  author={Yuan, Haobo and Li, Xiangtai and Zhang, Tao and Huang, Zilong and Xu, Shilin and Ji, Shunping and Tong, Yunhai and Qi, Lu and Feng, Jiashi and Yang, Ming-Hsuan},
  journal={arXiv preprint arXiv:2501.04001},
  year={2025}
}