moonshotai/WorldVQA
WorldVQA WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models HomePage | Dataset | Paper | Code Abstract We introduce WorldVQA, a benchmark designed to evaluate the atomic vision-centric world knowledge of Multimodal Large Language Models (MLLMs). Current evaluations often conflate visual knowledge retrieval with reasoning. In contrast, WorldVQA decouples these capabilities to strictly measure "what the… See the full description on the dataset page: https://huggingface.co/datasets/moonshotai/WorldVQA.
WorldVQA
WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models
<p align="center"> <a href="https://worldvqa2026.github.io/WorldVQA/"> HomePage</a> | <a href="https://huggingface.co/datasets/moonshotai/WorldVQA"> Dataset</a> | <a href="https://arxiv.org/abs/2602.02537v1"> Paper</a> | <a href="https://github.com/MoonshotAI/WorldVQA/"> Code</a> </p>
Abstract
We introduce WorldVQA, a benchmark designed to evaluate the atomic vision-centric world knowledge of Multimodal Large Language Models (MLLMs). Current evaluations often conflate visual knowledge retrieval with reasoning. In contrast, WorldVQA decouples these capabilities to strictly measure "what the model memorizes." The benchmark assesses the atomic capability of grounding and naming visual entities across a stratified taxonomy, spanning from common head-class objects to long-tail rarities. We expect WorldVQA serves as a rigorous test for visual factuality, thereby establishing a standard for assessing the encyclopedic breadth and hallucination rates of current and next-generation frontier models. <img src="images/main_figure.jpg">
Details
WorldVQA is a meticulously curated benchmark designed to evaluate atomic vision-centric world knowledge in Multimodal Large Language Models (MLLMs). The dataset comprises 3,000 VQA pairs across 8 categories, with careful attention to linguistic and cultural diversity.
Note: Due to copyright concerns, the "People" category has been removed from this release. The original benchmark contains 3,500 VQA pairs across 9 categories.
Leaderboard
Our evaluation reveals significant gaps in visual encyclopedic knowledge, with no model surpassing the 50% accuracy threshold.
We show a mini-leaderboard here and please find more information in our paper or homepage.
Overall Performance
The leaderboard below shows the overall performance on WorldVQA (first 8 categories, excluding "People" due to systematic refusal behaviors in closed-source models):
Citation
If you find WorldVQA useful for your research, please cite our work:
@misc{zhou2026worldvqameasuringatomicworld,
title={WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models},
author={Runjie Zhou and Youbo Shao and Haoyu Lu and Bowei Xing and Tongtong Bai and Yujie Chen and Jie Zhao and Lin Sui and Haotian Yao and Zijia Zhao and Hao Yang and Haoning Wu and Zaida Zhou and Jinguo Zhu and Zhiqi Huang and Yiping Bao and Yangyang Liu and Y. Charles and Xinyu Zhou},
year={2026},
eprint={2602.02537},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2602.02537},
}