Modelpublic

mayocream/RORem-mixed-GGUF

sourceHugging Faceopenrail++updated 2mo agoView on Hugging Face

5likes727downloads

Model Card

RORem-mixed GGUF (Q4_K)

This repository contains a Q4_K GGUF conversion of the RORem-mixed SDXL inpainting UNet for stable-diffusion.cpp.

RORem is an object-removal/inpainting model from RORem: Training a Robust Object Remover with Human-in-the-Loop. The mixed-resolution checkpoint was trained for both 512x512 and 1024x1024 inputs.

Files

File	Description
`rorem-mixed-unet-q4_K.gguf`	RORem-mixed UNet in GGML Q4_K format (1.885 GiB)
`sdxl-version-marker.safetensors`	Tiny metadata-only file used to identify the pipeline as SDXL Inpaint

This is not a complete SDXL pipeline. It does not include the VAE, CLIP-L, or CLIP-G weights. Use those components from `diffusers/stable-diffusion-xl-1.0-inpainting-0.1`.

stable-diffusion.cpp usage

The model was converted and tested with the official CUDA prebuilt from stable-diffusion.cpp release master-782-b290693.

bash

sd-cli \
  --model sdxl-version-marker.safetensors \
  --diffusion-model rorem-mixed-unet-q4_K.gguf \
  --vae /path/to/sdxl-inpainting/vae/diffusion_pytorch_model.fp16.safetensors \
  --clip_l /path/to/sdxl-inpainting/text_encoder/model.fp16.safetensors \
  --clip_g /path/to/sdxl-inpainting/text_encoder_2/model.fp16.safetensors \
  --init-img input.png \
  --mask mask.png \
  --prompt "clean manga illustration, crisp black line art, flat colors, seamless original background, clean white speech bubble, no text" \
  --negative-prompt "text, letters, words, symbols, watermark, signature, blurry, smudged, dirty, gray artifacts, extra objects, photorealistic" \
  --width 512 \
  --height 512 \
  --steps 30 \
  --cfg-scale 8 \
  --strength 0.999 \
  --sampling-method euler \
  --scheduler discrete \
  --fa \
  --output output.png

The white portion of the mask is regenerated. For manga translation, dilating the text or SFX mask before inference helps remove outlines and antialiasing remnants.

Local benchmark

Single 512x512 manga inpainting crop, 30 Euler steps, fixed seed, RTX 5090, stable-diffusion.cpp CUDA build b290693:

UNet format	Masked MAE vs FP16	Masked PSNR vs FP16	Sampling	End-to-end	Loaded parameter VRAM
Q4_K	1.1033	43.741 dB	4.06 s	6.383 s	3657.61 MB
Q4_0	1.8081	41.193 dB	4.03 s	6.348 s	3573.38 MB
FP16	reference	reference	5.15 s	7.536 s	6624.13 MB

This is one local sample, not a comprehensive quality benchmark. Q4K was effectively tied with Q40 in speed while producing an output closer to the FP16 reference.

Limitations

—RORem can hallucinate texture, line art, or objects inside large masks.
—Output quality depends strongly on mask dilation, crop context, resolution, prompt, and seed.
—This checkpoint is intended for inpainting/object removal, not general text-to-image generation.
—The quantized model may differ from FP16 output.

Attribution and licenses

—RORem project and original checkpoint: leeruibin/RORem, published by its authors under Apache 2.0.
—SDXL inpainting base: `diffusers/stable-diffusion-xl-1.0-inpainting-0.1`, licensed under CreativeML Open RAIL++-M.
—Inference runtime: stable-diffusion.cpp.

Users are responsible for complying with all applicable upstream licenses and usage restrictions.

Citation

bibtex

@article{li2024RORem,
  title={RORem: Training a Robust Object Remover with Human-in-the-Loop},
  author={Ruibin Li and Tao, Yang and Song, Guo and Lei, Zhang},
  year={2025},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition}
}