Modelpublic

je-suis-tm/dakota_johnson_lora_flux

sourceHugging Facemitupdated 6mo agoView on Hugging Face

1likes24downloads

Model Card

Dakota Johnson Lora Flux

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

This a non-quantized version of https://huggingface.co/je-suis-tm/dakota_johnson_lora_flux_nf4. Both are trained on the same dataset. The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux.py. The training took 3 hours on A100 80GB with max VRAM consumption at 35GB. The inference consumes 36GB of VRAM.

Train

export MODEL_NAME="black-forest-labs/FLUX.1-dev"
export INSTANCE_DIR="/pvol/dakota_johnson"
export OUTPUT_DIR="/pvol/dakota_johnson_lora_flux"
accelerate config default
accelerate launch train_dreambooth_lora_flux1.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --mixed_precision="bf16" \
  --dataset_name=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --gradient_checkpointing \
  --instance_prompt="Dakota Johnson" \
  --caption_column="text" \
  --resolution=1024 \
  --train_batch_size=1 \
  --guidance_scale=1 \
  --use_8bit_adam \
  --checkpointing_steps=100 \
  --gradient_accumulation_steps=4 \
  --optimizer="adamW" \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=100 \
  --max_train_steps=1500 \
  --rank=4 \
  --seed="0"

Usage

python

import torch
from diffusers import FluxPipeline

device = "cuda:0"

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)

pipe.load_lora_weights("je-suis-tm/dakota_johnson_lora_flux",
                       weight_name='pytorch_lora_weights.safetensors')


prompt = "Glacier beauty. Beautiful colors. Dakota Johnson stands on a frozen lake, dressed in a dvr dulcesa onepiece made of ral kntarmr fabric, radiating a mysterious allure. The open knit design over her toned stomach reveals fragments of skin, allowing icy light to shine through. She has long straight hair as she stares intently into the camera, the reflection of the glaciers creating a surreal mirror effect."

image = pipe(
  prompt=prompt,
  generator=torch.Generator(device=device).manual_seed(42),
  num_inference_steps=50, # 28 is a good trade-off
  guidance_scale=4,
  height=1024, 
  width=1024,
).images[0]

image.save("dakota_johnson.png")

Trigger words

You should use Dakota Johnson to trigger the image generation.

Download model

Download them in the Files & versions tab.