Datasetpublic

Chrisyichuan/moca-colpali-training

Chrisyichuan/moca-colpali-training MOCA ColPali contrastive training data with hard negatives. Contents moca_colpali_converted.jsonl — query-image pairs with hard negatives images/ — all referenced images Each metadata row: { "query": "...", "chunk_path": "images/...", "neg_chunk_paths": ["images/...", "images/..."], "source_positive_rank": 0, "source_positive_score": 0.0, "source_dataset": "moca" } Summary rows: 118195 unique… See the full description on the dataset page: https://huggingface.co/datasets/Chrisyichuan/moca-colpali-training.

sourceHugging Facemitupdated 6mo agoView on Hugging Face

1likes25downloads

Dataset Card

Chrisyichuan/moca-colpali-training

MOCA ColPali contrastive training data with hard negatives.

—moca_colpali_converted.jsonl — query-image pairs with hard negatives
—images/ — all referenced images

Each metadata row:

json

{
  "query": "...",
  "chunk_path": "images/...",
  "neg_chunk_paths": ["images/...", "images/..."],
  "source_positive_rank": 0,
  "source_positive_score": 0.0,
  "source_dataset": "moca"
}

Summary

—rows: 118195
—unique images: 118195
—avg negatives/row: 2.00

Download

python

from huggingface_hub import snapshot_download
snapshot_download(repo_id="Chrisyichuan/moca-colpali-training", repo_type="dataset", local_dir="data/moca-colpali-training")

Image Storage

Images are stored as 119 tar shards under image_shards/ for fast download.

After cloning/downloading, extract images:

bash

python extract_hf_image_shards.py --dataset-dir .

This creates images/ with all referenced image files.