zamasW/drug_resized_LoRA

<!-- This model card has been generated automatically according to the information the training script had access to. You should probably proofread and complete it, then remove this comment. -->

SDXL LoRA DreamBooth - zamasW/drugresizedLoRA

<Gallery />

Model description

These are zamasW/drugresizedLoRA LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.

The weights were trained using DreamBooth.

LoRA for the text encoder was enabled: False.

Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.

Trigger words

You should use a face after drug abuse to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Intended uses & limitations

How to use

import torch
from diffusers import DiffusionPipeline, AutoencoderKL
repo_id = 'zamasW/drug_resized_LoRA'
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    vae=vae,
    torch_dtype=torch.float16,
    variant="fp16",
    use_safetensors=True
)
pipe.load_lora_weights(repo_id)
_ = pipe.to("cuda")

prompt = "a face after drug abuse" 

image = pipe(prompt=prompt, num_inference_steps=25).images[0]

Limitations and bias

coming soon

Training details

Dataset : over 50 images of 1080*1080 of drug abused faces

• —Model: stabilityai/stable-diffusion-xl-base-1.0
• —VAE: madebyollin/sdxl-vae-fp16-fix
• —Instance Prompt: "a face after dug abuse"
• —Image Resolution: 1024 x 1024
• —Training Batch Size: 1
• —Gradient Accumulation Steps: 3
• —Gradient Checkpointing: Enabled
• —Learning Rate: 1e-4
• —SNR Gamma: 5.0
• —LR Scheduler: constant
• —Warmup Steps: 0
• —Precision: fp16 (mixed precision)
• —Optimizer: 8-bit Adam
• —Max Train Steps: 500
• —Checkpointing Steps: 717
• —Seed: 0