Modelpublic

wookiekim/SD3.5M-SOLACE

sourceHugging Facemitupdated 4mo agoView on Hugging Face

0likes7downloads

Model Card

SD3.5M-SOLACE

LoRA adapter from SOLACE (Self-cOnfidence reward for aLigning text-to-imAge models via ConfidencE optimization), CVPR 2026.

SOLACE applied to Stable Diffusion 3.5 Medium, using the model's own denoising confidence as an intrinsic reward (no external reward model at training time).

—Base model: `stabilityai/stable-diffusion-3.5-medium`
—Method: SOLACE intrinsic self-confidence reward (built on Flow-GRPO)
—Code: https://github.com/wookiekim/SOLACE
—Adapter type: PEFT LoRA (rank 32, applied to the SD3 transformer attention projections)

Usage

python

import torch
from diffusers import StableDiffusion3Pipeline
from peft import PeftModel

model_id = "stabilityai/stable-diffusion-3.5-medium"
lora_ckpt_path = "wookiekim/SD3.5M-SOLACE"
device = "cuda"

# Load base model and apply the SOLACE LoRA adapter
pipe = StableDiffusion3Pipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, lora_ckpt_path)
pipe.transformer = pipe.transformer.merge_and_unload()
pipe = pipe.to(device)

prompt = "a photo of a cat wearing a small red hat"
image = pipe(
    prompt,
    height=512,
    width=512,
    num_inference_steps=40,
    guidance_scale=4.5,
    negative_prompt="",
).images[0]
image.save("solace.png")

Citation

bibtex

@inproceedings{kim2026solace,
    title={Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards},
    author={Kim, Wookyoung and others},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2026}
}

Acknowledgments

This work builds upon Flow-GRPO by Jie Liu et al.