MMVP/MMVP_VLM
MMVP-VLM Benchmark Datacard Basic Information Title: MMVP-VLM Benchmark Description: The MMVP-VLM (Multimodal Visual Patterns - Visual Language Models) Benchmark is designed to systematically evaluate the performance of recent CLIP-based models in understanding and processing visual patterns. It distills a subset of questions from the original MMVP benchmark into simpler language descriptions, categorizing them into distinct visual patterns. Each visual pattern is… See the full description on the dataset page: https://huggingface.co/datasets/MMVP/MMVP_VLM.
MMVP-VLM Benchmark Datacard
Basic Information
Title: MMVP-VLM Benchmark
Description: The MMVP-VLM (Multimodal Visual Patterns - Visual Language Models) Benchmark is designed to systematically evaluate the performance of recent CLIP-based models in understanding and processing visual patterns. It distills a subset of questions from the original MMVP benchmark into simpler language descriptions, categorizing them into distinct visual patterns. Each visual pattern is represented by 15 text-image pairs. The benchmark assesses whether CLIP models can accurately match these image-text combinations, providing insights into the capabilities and limitations of these models.
Dataset Details
- Content Types: Text-Image Pairs
- Volume: Balanced number of questions for each visual pattern, with each pattern represented by 15 pairs.
- Source of Data: Subset from MMVP benchmark, supplemented with additional questions for balance
- Data Collection Method: Distillation and categorization of questions from MMVP benchmark into simpler language
Usage
Intended Use
- Evaluation of CLIP models' ability to understand and process various visual patterns.