nihalChourasiya/CDCIR

CDCIR — Cross-Domain Composed Image Retrieval

Overview

CDCIR (Cross-Domain Composed Image Retrieval) is a deep-learning system for composed image retrieval (CIR) — a task where the user provides a reference image together with a text modification (e.g., "make it red and shorter"), and the system retrieves images from a gallery that match the combined intent. Unlike traditional image retrieval (query by image) or text-based search (query by keywords), CIR jointly reasons over visual and textual inputs to compose a unified query embedding and find the best-matching target images.

The key architectural innovation is the TextQueryDINOCombiner: a lightweight, trainable fusion module that bridges two powerful frozen foundation models — OpenCLIP (for semantic text and image understanding) and DINOv2 (for fine-grained spatial visual features) — to produce a composed query representation that outperforms either model alone.

The project ships as a self-contained Gradio web application with pre-trained weights, pre-computed gallery features, and a validation image set, making it ready to demo out-of-the-box.

Tech Stack

Installation & Usage

Prerequisites

• —Python 3.8+
• —CUDA-capable GPU recommended (but CPU is supported)

Setup

# Clone the repository (ensure Git LFS is installed for large files)
git lfs install
git clone https://github.com/nihalChourasiya/CDCIR.git
cd CDCIR

# Install dependencies
pip install -r requirements.txt

Run the Demo

python app.py

This will:

1. 1.Load the pre-trained combiner from eval_bundle.pt
2. 2.Initialize CLIP and DINOv2 foundation models
3. 3.Build the retrieval gallery from val_clip_cls.pt and val_images/
4. 4.Launch a Gradio web interface (typically at http://localhost:7860)

Using the Interface

1. 1.Upload a reference image (any format — it will be converted to RGB)
2. 2.Type a text modification describing how you want to change the reference
3. 3.Adjust the top-K slider to control how many results to show (1–50)
4. 4.Click Search or press Enter to retrieve matching images from the gallery

Project Structure

CDCIR/
├── app.py                  # Main application: initialization, inference, Gradio UI
├── models.py               # TextQueryDINOCombiner architecture (trainable module)
├── feature_extraction.py   # CLIP and DINOv2 feature extraction pipelines
├── retrieval.py            # Gallery management and cosine-similarity retrieval
├── requirements.txt        # Python dependencies
├── eval_bundle.pt          # Pre-trained combiner weights + training config (~30MB)
├── val_clip_cls.pt         # Pre-computed gallery CLIP CLS embeddings (~124MB)
├── val_images/             # Validation gallery images (~15,400 images)
│   ├── 00/
│   ├── 01/
│   ├── 02/
│   └── 03/
├── .gitattributes          # Git LFS tracking rules for large files
└── README.md               # This file