oindrila13saha/sigma-gen-lora

<h1 style="font-size: 2.5em; text-align: center; margin-bottom: 0.2em;">SIGMA-Gen: Structure and Identity Guided Multi-subject Assembly for Image Generation</h1>

<p align="center"> <a href="https://oindrilasaha.github.io"><b>Oindrila Saha</b></a> · <a href="https://research.adobe.com/person/vojtech-krs/"><b>Vojtech Krs</b></a> · <a href="https://research.adobe.com/person/radomir-mech/"><b>Radomir Mech</b></a> · <a href="https://people.cs.umass.edu/~smaji/"><b>Subhransu Maji</b></a> · <a href="https://kmatzen.com"><b>Kevin Blackburn-Matzen</b></a> · <a href="http://mgadelha.me"><b>Matheus Gadelha</b></a> </p>

<p align="center"> <a href="https://arxiv.org/abs/2510.06469"><img src="https://img.shields.io/badge/arXiv-2510.06469-red?color=%23aa1a1a" alt="arXiv"></a> <a href="https://oindrilasaha.github.io/SIGMA-Gen/"><img src="https://img.shields.io/badge/Project%20Page-SIGMA--Gen-cyan?color=%23cbe6f2" alt="Project Page"></a> <a href="https://iclr.cc/"><img src="https://img.shields.io/badge/ICLR-2026-blue?color=%234a90d9" alt="ICLR 2026"></a> </p>

<p align="center"><b>University of Massachusetts Amherst &nbsp; | &nbsp; Adobe Research</b></p>

Overview

SIGMA-Gen enables multi-identity image generation in a single pass, guided by structural and spatial constraints. It supports varied user guidance precision — from 2D/3D boxes to pixel-level segmentations and depth — achieving state-of-the-art performance in identity preservation, generation quality, and processing speed.

Weights

This repository contains two LoRA adapters trained on top of FLUX.1-Kontext-dev:

Usage

from diffusers.pipelines import FluxKontextPipeline
import torch

pipe = FluxKontextPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16
).to("cuda")

pipe.load_lora_weights("oindrila13saha/sigma-gen-lora", weight_name="cond1.safetensors", adapter_name="cond1")
pipe.load_lora_weights("oindrila13saha/sigma-gen-lora", weight_name="cond2.safetensors", adapter_name="cond2")
pipe.set_adapters(["cond1", "cond2"], adapter_weights=128)

Citation

@article{saha2025sigma,
  title={SIGMA-GEN: Structure and Identity Guided Multi-subject Assembly for Image Generation},
  author={Saha, Oindrila and Krs, Vojtech and Mech, Radomir and Maji, Subhransu and Blackburn-Matzen, Kevin and Gadelha, Matheus},
  journal={arXiv preprint arXiv:2510.06469},
  year={2025}
}