Datasetpublic

glab-caltech/FGVQA

FGVQA This repository contains the FGVQA benchmark suite introduced in the paper Same or Not? Enhancing Visual Perception in Vision-Language Models.FGVQA contains 12,000 challenging (image, question, answer) tuples emphasizing fine-grained image understanding. The benchmark suite is composed of six sub-benchmarks: TWIN-eval ILIAS Google Landmarks v2 MET CUB Inquire For evaluating on the dataset with LMMS-eval, please refer to this repo. Citation If you use the… See the full description on the dataset page: https://huggingface.co/datasets/glab-caltech/FGVQA.

sourceHugging Facecc-by-nc-4.0updated 9mo agoView on Hugging Face

2likes56downloads

Dataset Card

FGVQA

This repository contains the FGVQA benchmark suite introduced in the paper Same or Not? Enhancing Visual Perception in Vision-Language Models.FGVQA contains 12,000 challenging (image, question, answer) tuples emphasizing fine-grained image understanding.

The benchmark suite is composed of six sub-benchmarks: 1) TWIN-eval 2) ILIAS 3) Google Landmarks v2 4) MET 5) CUB 6) Inquire

For evaluating on the dataset with LMMS-eval, please refer to this repo.

Citation

If you use the FGVQA benchmark suite in your research, please use the following BibTeX entry.

@misc{marsili2025notenhancingvisualperception,
      title={Same or Not? Enhancing Visual Perception in Vision-Language Models}, 
      author={Damiano Marsili and Aditya Mehta and Ryan Y. Lin and Georgia Gkioxari},
      year={2025},
      eprint={2512.23592},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.23592}, 
}