GenIntelLab/SOCO-LVLM

1---2license: cc-by-4.03pretty_name: SOCO-LVLM4viewer: false5task_categories:6- visual-question-answering7tags:8- computer-vision9- multimodal10- object-correspondence11- synthetic-data12---13 14# SOCO-LVLM15 16SOCO-LVLM provides multiple-choice semantic object correspondence evaluation data for17LVLMs. This is the SOCO-LVLM v1 release, derived from SOCOv1. The original SOCO18correspondence benchmark is available in19the [GenIntelLab/SOCO](https://huggingface.co/datasets/GenIntelLab/SOCO) dataset repository.20 21## Repository Layout22 23```text24GenIntelLab/SOCO-LVLM25  SOCO_LVLM/26    soco_lvlm_img.tsv27    soco_lvlm_imgtxt.tsv28    soco_lvlm_txt.tsv29  README.md30```31 32## Variants33 34- `soco_lvlm_img.tsv`: image-input evaluation variant (approximately 3.24 GB).35- `soco_lvlm_imgtxt.tsv`: image-and-text evaluation variant (approximately 3.24 GB).36- `soco_lvlm_txt.tsv`: text-input evaluation variant (approximately 1.63 GB).37 38Each TSV uses the columns `question`, `image`, `image_path`, `answer`, `index`, `g_index`,39`qid`, `category`, `A`, `B`, `C`, and `D`.40 41## Download42 43Install the Hub client:44 45```bash46pip install -U huggingface_hub47```48 49Download all three variants:50 51```bash52hf download GenIntelLab/SOCO-LVLM --repo-type dataset --local-dir SOCO-LVLM53```54 55Download only one variant in Python:56 57```python58from huggingface_hub import hf_hub_download59 60path = hf_hub_download(61    repo_id="GenIntelLab/SOCO-LVLM",62    repo_type="dataset",63    filename="SOCO_LVLM/soco_lvlm_img.tsv",64)65```66 67Replace the filename with `soco_lvlm_imgtxt.tsv` or `soco_lvlm_txt.tsv` to select a68different evaluation variant.69 70## Citation71 72```bibtex73@misc{duenkel2026soco,74  title         = {SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models},75  author        = {D{\"u}nkel, Olaf and Sunagad, Basavaraj and Wang, Haoran and76                   Hoffmann, David T. and Theobalt, Christian and Kortylewski, Adam},77  year          = {2026},78  eprint        = {2605.31597},79  archivePrefix = {arXiv},80  primaryClass  = {cs.CV},81  url           = {https://arxiv.org/abs/2605.31597}82}83```84