tsunghanwu/reverse_llava_v15

REVERSE-v1.5-7B

<a href="https://arxiv.org/abs/2504.13169"> <img src="https://img.shields.io/badge/arXiv-2504.13169-b31b1b.svg" alt="arXiv" /> </a>

Model Summary

REVERSE-v1.5-7B is a novel open-source vision-language model (VLM) that performs both next-token predictioin and self-verification / self-correction during the generation process. Built on top of LLaVA-v1.5-7B, it is fine-tuned using the REVERSE Visual Instruct 1.3M dataset and equipped with a retrospective resampling mechanism that allows it to detect and correct hallucinations during generation. The model is trained in early March, 2025.

Performance

REVERSE achieves state-of-the-art hallucination reduction across a wide range of captioning and open-ended visual question answering benchmarks:

It also performs competitively on discriminative tasks compared with the base VLM.

Usage

Please refer to the installation guide on GitHub to get started: 👉 Installation Guide

Additional Resources

• —📄 Project Page: https://reverse-vlm.github.io/
• —🧾 Dataset: REVERSE Visual Instruct 1.3M
• —🔧 Ask Questions: GitHub Issues

Intended Use

Primary Use Cases:

• —Reducing hallucination in image captioning and VQA tasks
• —Benchmarking hallucination-aware generation
• —Research on grounded vision-language generation and self-correction

Target Users: Researchers, developers, and students working in computer vision, NLP, and multimodal AI.