rippleripple/MMIR

Paper: Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models Paper: https://arxiv.org/pdf/2502.16033 Website: https://jackie-2000.github.io/mmir.github.io/ MMIR: Specifically built to rigorously test MLLMs’ ability to detect real-world cross-modal inconsistencies. Contains 534 carefully curated samples, each injecting a single semantic mismatch across 5 error categories. It spans webpages, slides, and posters with diverse layouts, challenging models to… See the full description on the dataset page: https://huggingface.co/datasets/rippleripple/MMIR.