Modelpublic

juliensimon/stable-diffusion-v1-5-pokemon-lora

sourceHugging Facecreativeml-openrail-mupdated 6mo agoView on Hugging Face

4likes70downloads

Model Card

Stable Diffusion v1.5 — Pokemon LoRA

LoRA adaptation weights for Stable Diffusion v1.5, fine-tuned on the Pokemon BLIP Captions dataset (833 samples) to generate Pokemon-style images from text prompts.

Video walkthrough: Fine-tune Stable Diffusion with LoRA for as low as $1

Training Details

Detail	Value
Method	4-bit QLoRA following the official LoRA blog post
Hardware	Amazon EC2 g4dn.xlarge (1x NVIDIA T4 GPU)
Environment	AWS Deep Learning AMI for PyTorch
Training time	~6 hours
Cost	~$3 on-demand / ~$1 with EC2 Spot Instances

Usage

python

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16
)
pipe.unet.load_attn_procs("juliensimon/stable-diffusion-v1-5-pokemon-lora")
pipe.to("cuda")

image = pipe("a pokemon with blue fire wings").images[0]

Resources

—Training script and training logs are included in the repository.