CoolFace
Datasetpublic

VDBBench/multimodal-embedding-10M

Multimodal Embedding 10M Benchmark Dataset A large-scale vector search benchmark dataset containing 10M base vectors and 10K query vectors with pre-computed ground truth (top-100 nearest neighbors by Inner Product), generated from multimodal (image + text) inputs. Dataset Summary Property Value Base Vectors 10,000,000 Query Vectors 10,000 Dimension 4,096 Distance Metric Inner Product (IP) Top-K Ground Truth 100 Vector dtype float32 Embedding… See the full description on the dataset page: https://huggingface.co/datasets/VDBBench/multimodal-embedding-10M.

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes239downloads
Dataset Card

Multimodal Embedding 10M Benchmark Dataset

A large-scale vector search benchmark dataset containing 10M base vectors and 10K query vectors with pre-computed ground truth (top-100 nearest neighbors by Inner Product), generated from multimodal (image + text) inputs.

Dataset Summary

PropertyValue
Base Vectors10,000,000
Query Vectors10,000
Dimension4,096
Distance MetricInner Product (IP)
Top-K Ground Truth100
Vector dtypefloat32
Embedding ModelQwen3-VL-Embedding-8B
Source Datapixparse/cc12m-wds (Conceptual Captions 12M)
Base/Query OverlapYes (0.1%, query shards 40-41 included in base)

Splits

SplitRowsDescription
train10,000,000Base vectors (20 parquet shards, 500K rows each)
test10,000Query vectors
neighbors10,000Ground truth: top-100 nearest neighbor IDs per query

Schema

train / test: | Column | Type | Description | |--------|------|-------------| | id | int64 | Sequential identifier (0-indexed) | | emb | list\<float32\> | 4096-dim L2-normalized embedding |

neighbors: | Column | Type | Description | |--------|------|-------------| | id | int64 | Query ID (matches test split) | | neighbors | list\<int64\> | Top-100 base vector IDs by IP score (descending) |

Usage

python
from datasets import load_dataset
import numpy as np

ds = load_dataset("WenxingZhu/multimodal-embedding-10M")

# Base and query embeddings
base_emb = np.array(ds["train"]["emb"])    # (10_000_000, 4096) - streams from 20 shards
query_emb = np.array(ds["test"]["emb"])    # (10_000, 4096)

# Ground truth neighbors
neighbors = ds["neighbors"]["neighbors"]   # list of 10K lists, each 100 int64 IDs

Streaming (recommended for 10M)

python
from datasets import load_dataset

ds = load_dataset("WenxingZhu/multimodal-embedding-10M", split="train", streaming=True)
for row in ds:
    emb = row["emb"]  # list of 4096 floats
    break

Generation Details

  • —Model: Qwen3-VL-Embedding-8B via vLLM (pooling runner, bfloat16)
  • —Input: Image-text pairs from cc12m-wds WebDataset shards, streamed from HuggingFace
  • —Normalization: L2 normalized post-embedding
  • —Hardware: 6x NVIDIA A100-80GB (DGX)
  • —Throughput: ~25 samples/sec per GPU
  • —Total Generation Time: ~3 days for 10M embeddings
  • —Ground Truth: Brute-force inner product, chunked numpy computation

Related

License

Apache 2.0