datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MMEB_Test_InstructVideo-MME-v2
🔥 News
2026.06.11 Videos re-encoded to H265, maintaining consistent evaluation scores. Fixed 2 incorrect MP4s & 3 mismatched URLs. Original data preserved in the original branch.
2026.05.22 Task types are now available for Q1-Q3 in coherence (logic) groups.
🤗 About This Repo
This repository contains annotation data for "Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding". It mainly consists of three… See the full description on the dataset page: https://huggingface.co/datasets/MME-Benchmarks/Video-MME-v2.MMEB-eval
Massive Multimodal Embedding Benchmark
We compile a large set of evaluation tasks to understand the capabilities of multimodal embedding models. This benchmark covers 4 meta tasks and 36 datasets meticulously selected for evaluation.
The dataset is published in our paper VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks.
Dataset Usage
For each dataset, we have 1000 examples for evaluation. Each example contains a query and a set of… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MMEB-eval.MMEB-train
Massive Multimodal Embedding Benchmark
The training data split used for training VLM2Vec models in the paper VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025).
MMEB benchmark covers 4 meta tasks and 36 datasets meticulously selected for evaluating capabilities of multimodal embedding models.
During training, we utilize 20 out of the 36 datasets.
For evaluation, we assess performance on the 20 in-domain (IND) datasets and the remaining 16… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MMEB-train.MMEB-V3
MMEB-V3: Measuring the Performance Gaps of Omni-Modality Embedding Models
🌐 Website |
GitHub |
🏆 Leaderboard |
📖 MMEB-V3 Paper |
📖 MMEB-V2 Paper |
📖 MMEB-V1 Paper |
🤗 Models
Introduction
MMEB-V3 is a comprehensive benchmark for evaluating omni-modality embedding models across text, image, video, audio, visual-document, and agent-centric retrieval scenarios.
Building upon MMEB-V1 and MMEB-V2, MMEB-V3 adds 111 new tasks, resulting in 190 evaluation tasks in… See the full description on the dataset page: https://huggingface.co/datasets/VLM2Vec/MMEB-V3.MMEB_train_with_imagemmE5-MMEB-hardneg
mmE5 Labeled Data
This dataset contains datasets used for the supervised finetuning of mmE5 (mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data):
MMEB (with hard negative)
InfoSeek (from M-BEIR)
TAT-DQA
ArxivQA
Github
Image Preparation
First, you should prepare the images used for training:
Image Downloads
Download All Images Used in mmE5:
You can use the script provided in our source code to download all images used… See the full description on the dataset page: https://huggingface.co/datasets/intfloat/mmE5-MMEB-hardneg.MME-Benchmark-pt
Avaliação - MME-Perception
Estrutura do Diretório
main
├── MME_Benchmark
│ ├── artwork
│ │ ├── images
│ │ │ ├── 1.jpg
│ │ │ ├── 2.jpg
│ │ │ ├── ...
│ │ ├── question_answers_YN
│ │ │ ├── 1.txt
│ │ │ ├── 2.txt
│ │ │ ├── ...
│ ├── celebrity
│ ├── code_reasoning
│ ├── ...
├── calculation.py
├── translate_MME
Estrutura dos Arquivos TXT
Cada arquivo num.txt contém as perguntas correspondentes à imagem num.jpg.… See the full description on the dataset page: https://huggingface.co/datasets/LucasLima/MME-Benchmark-pt.BToks-MMEB-train
BToks MMEB-train
This dataset repository contains Lance-format converted data used by the open-source reproduction code for Bottleneck Tokens for Unified Multimodal Retrieval (arXiv:2604.11095).
Source
Converted from TIGER-Lab/MMEB-train. This repository does not change upstream ownership, licensing, citation requirements, or usage restrictions.
Format
The data is stored as Lance tables for the BToks/VLM2Emb training and evaluation scripts. The… See the full description on the dataset page: https://huggingface.co/datasets/siyrus/BToks-MMEB-train.MMEB-train-fixed
MMEB train split used in MoCa Continual Pre-training
🏠 Homepage | 💻 Code | 🤖 MoCa-Qwen25VL-7B | 🤖 MoCa-Qwen25VL-3B | 📚 Datasets | 📄 Paper
Introduction
This is a interleaved multimodal pre-training dataset used in the modality-aware continual pre-training of MoCa models. It is adapted from the train split of
MMEB by concatenating queries and positive documents.
The dataset consists of interleaved multimodal examples. text is a string containing text while images… See the full description on the dataset page: https://huggingface.co/datasets/moca-embed/MMEB-train-fixed.Nyx-mmE5-MMEBTIGER-Lab_MMEB-train
MMEB Training Dataset (Lance Format)
This is a Lance-format version of the TIGER-Lab/MMEB-train dataset, optimized for efficient storage and fast random access.
The original dataset is used for training VLM2Vec models in the paper VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025).
Directory Structure
TIGER-Lab_MMEB-train/
└── data/
├── A-OKVQA/
│ ├── train.lance
│ ├── original.lance
│ └── diverse.lance… See the full description on the dataset page: https://huggingface.co/datasets/laughatwill/TIGER-Lab_MMEB-train.MMEB-eval
Massive Multimodal Embedding Benchmark
We compile a large set of evaluation tasks to understand the capabilities of multimodal embedding models. This benchmark covers 4 meta tasks and 36 datasets meticulously selected for evaluation.
The dataset is published in our paper VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks.
Dataset Usage
For each dataset, we have 1000 examples for evaluation. Each example contains a query and a set of targets.… See the full description on the dataset page: https://huggingface.co/datasets/Y-J-Ju/MMEB-eval.MMEB_Test_5KMMEB_5KMMEB-train
Massive Multimodal Embedding Benchmark
The training data split used for training VLM2Vec models in the paper VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025).
MMEB benchmark covers 4 meta tasks and 36 datasets meticulously selected for evaluating capabilities of multimodal embedding models.
During training, we utilize 20 out of the 36 datasets.
For evaluation, we assess performance on the 20 in-domain (IND) datasets and the remaining 16… See the full description on the dataset page: https://huggingface.co/datasets/Y-J-Ju/MMEB-train.MMEB-eval-ChartQA-beir-v3MMEB-eval-ChartQA-beirMMEB-eval-DocVQA-beirMMEB-eval-Wiki-SS-NQ-beir-v3MMEB-eval-VisDial-beirMMEB-eval-OVEN-beir-v3MMEB-eval-A-OKVQA-beirMMEB-eval-MSCOCO_i2t-beirMMEB-eval-ObjectNet-beirMMEB-eval-GQA-beir-v2MMEB_Pro_Video_RetrievalMMEB-eval-OK-VQA-beirMMEB-eval-ImageNet-R-beir-v3MMEB-eval-FashionIQ-beir-v3