lightonai/colpali-train-images
ColPali Train Images This dataset contains the document-page images used by lightonai/colpali-train-fine-tuning. The pair is a reformatted derivative of vidore/colpali_train_set for multimodal retrieval fine-tuning. Dataset structure The train split contains: Column Type Description image_filename string Stable key used by the companion fine-tuning dataset. image image Document-page image. Load the dataset from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/colpali-train-images.
ColPali Train Images
This dataset contains the document-page images used by lightonai/colpali-train-fine-tuning. The pair is a reformatted derivative of vidore/colpali_train_set for multimodal retrieval fine-tuning.
Dataset structure
The train split contains:
Load the dataset
from datasets import load_dataset
images = load_dataset(
"lightonai/colpali-train-images",
split="train",
)Join image_filename with the same column in the documents config of lightonai/colpali-train-fine-tuning.
Source and license
This is a reformatted derivative of vidore/colpali_train_set. The upstream dataset combines several academic datasets and synthetic data. Review the upstream dataset card and the licenses of its constituent sources before redistribution or commercial use.
Citation
Please cite the original ColPali work:
@misc{faysse2024colpaliefficientdocumentretrieval,
title={ColPali: Efficient Document Retrieval with Vision Language Models},
author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Celine Hudelot and Pierre Colombo},
year={2024},
eprint={2407.01449},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2407.01449}
}