COinCO/Qwen2.5-VL-3B-Location

Qwen2.5-VL-3B Location Context Classifier

Authors: Tianze Yang\, Tyson Jordan\, Ruitong Sun\, Ninghao Liu, Jin Sun \Equal contribution | Affiliation: University of Georgia

Overview

A fine-tuned Qwen2.5-VL-3B-Instruct model for detecting out-of-context objects based on the location criterion.

Given an image with an object marked by a red bounding box, the model evaluates whether the object is placed in a physically and contextually reasonable position — e.g., supported by a surface, on the ground, or in a plausible environment. Objects floating in mid-air or embedded in other objects are classified as out-of-context.

How to Use

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
import torch

model_id = "COinCO/Qwen2.5-VL-3B-Location"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

Training Details

Evaluation Results

Inpainted Test Set (In-context vs Out-of-context)

Real COCO Images (shortcut learning detection, higher = better)

The baseline model heavily relies on visual artifacts (shortcut learning), achieving only 47.55% on real images. After fine-tuning, the model correctly identifies 91.35% of real objects as in-context, demonstrating significantly reduced shortcut reliance.

Related Resources

• —Dataset: COinCO/COinCO-dataset
• —Code: YangTianze009/COinCO
• —Other models: Co-occurrence | Size

Citation

@article{yang2025coinco,
  title={Common Inpainted Objects In-N-Out of Context},
  author={Tianze Yang and Tyson Jordan and Ruitong Sun and Ninghao Liu and Jin Sun},
  year={2025}
}