CoolFace
Datasetpublic

VDBBench/multimodal-embedding-1M

Multimodal Embedding 1M Benchmark Dataset A vector search benchmark dataset containing 1M base vectors and 10K query vectors with pre-computed ground truth, generated from multimodal (image + text) inputs. Dataset Description Each embedding is produced by encoding an image-text pair into a single 4096-dimensional vector using Qwen3-VL-Embedding-8B, a state-of-the-art multimodal embedding model. Source data: pixparse/cc3m-wds (Conceptual Captions 3M in WebDataset… See the full description on the dataset page: https://huggingface.co/datasets/VDBBench/multimodal-embedding-1M.

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes104downloads
Dataset Card

Multimodal Embedding 1M Benchmark Dataset

A vector search benchmark dataset containing 1M base vectors and 10K query vectors with pre-computed ground truth, generated from multimodal (image + text) inputs.

Dataset Description

Each embedding is produced by encoding an image-text pair into a single 4096-dimensional vector using Qwen3-VL-Embedding-8B, a state-of-the-art multimodal embedding model.

  • —Source data: pixparse/cc3m-wds (Conceptual Captions 3M in WebDataset format)
  • —Embedding model: Qwen3-VL-Embedding-8B via vLLM (runner="pooling", dtype=bfloat16)
  • —Input format: Each sample consists of an image and its corresponding caption, embedded together as a multimodal input
  • —Normalization: L2 normalized

Files

FileRowsColumnsDescription
train.parquet1,000,000id (int64), emb (list\<float32\>)Base vectors from shards 0-199
test.parquet10,000id (int64), emb (list\<float32\>)Query vectors from shards 200-201
neighbors.parquet10,000id (int64), neighbors (list\<int64\>)Ground truth: top-100 nearest neighbors by Inner Product

Key Properties

PropertyValue
Dimension4096
Distance metricInner Product (IP)
Base size1,000,000
Query size10,000
Top-K (ground truth)100
Vector dtypefloat32
Base/Query overlapNone (disjoint shards)

Usage

python
from datasets import load_dataset

# Load base and query splits
ds = load_dataset("WenxingZhu/multimodal-embedding-1M")
train = ds["train"]  # 1M base vectors
test = ds["test"]    # 10K query vectors

# Load ground truth
import pyarrow.parquet as pq
gt = pq.read_table("neighbors.parquet").to_pandas()

# Access embeddings
import numpy as np
base_emb = np.array(train["emb"])      # (1000000, 4096)
query_emb = np.array(test["emb"])      # (10000, 4096)
neighbors = np.array(gt["neighbors"].tolist())  # (10000, 100)

Generation Details

  • —Hardware: 4x NVIDIA A100-80GB (DGX)
  • —Inference: vLLM v0.17.1, pooling mode, batch size 16
  • —Throughput: ~25 samples/sec per GPU
  • —Total time: ~2.75 hours for 1M embeddings
  • —Prompt template: System message "Represent the user's input." + User message with image and text content

License

This dataset is released under the Apache 2.0 license. The source images and captions are from Conceptual Captions 3M.