Modelpublic

AlekseyCalvin/StabledHSTorY_SD3.5_LoRA_V1

sourceHugging Facecreativeml-openrail-mupdated 2y agoView on Hugging Face

0likes784downloads

Model Card

HSTsd3ii

Model trained with AI Toolkit by Ostris <Gallery />

Trigger words

'HST style autochrome photo'

Config Parameters

Dim:16 Alpha:32 Optimizer:Ademamix8bit LR:2e-4 More info below! Fine-tuned using the Google Colab Notebook of ai-toolkit. I've used A100 via Colab Pro. However, training SD3.5 may potentially work with Free Colab or lower VRAM in general: Especially if one were to use: ...Say, lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the /lowvram, true/ argument, and plausibly specifying alternate quantization variants. Generally, VRAM expenditures for fine-tuning SD3.5 tend to be lower than for Flux during training. So, try it! To use on Colab*, modify a Flux template Notebook from here with parameters from Ostris' example config for SD3.5 here!

job: extension
config:
  name: HSTsd3ii
  process:
  - type: sd_trainer
    training_folder: /content/drive/MyDrive/HSTsd3ii
    performance_log_every: 600
    device: cuda:0
    network:
      type: lora
      linear: 16
      linear_alpha: 32
    save:
      dtype: float16
      save_every: 250
      push_to_hub: true
      hf_repo_id: AlekseyCalvin/HSTsd3iii
      hf_private: true
      max_step_saves_to_keep: 16
    datasets:
    - folder_path: /content/dataset
      caption_ext: txt
      caption_dropout_rate: 0.0
      shuffle_tokens: false
      cache_latents_to_disk: true
      resolution:
      - 1024
    train:
      batch_size: 4
      steps: 4000
      gradient_accumulation_steps: 1
      train_unet: true
      train_text_encoder: false
      gradient_checkpointing: true
      noise_scheduler: flowmatch
      timestep_type: linear
      optimizer: ademamix8bit
      lr: 0.0002
      skip_first_sample: true
      ema_config:
        use_ema: true
        ema_decay: 0.8
      dtype: bf16
    model:
      name_or_path: stabilityai/stable-diffusion-3.5-large
      is_v3: true
      quantize: true

Plus validation settings: Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives.

Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc.

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Use it with the 🧨 diffusers library

from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3.5-large', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('AlekseyCalvin/HSTsd3iii', weight_name='HSTsd3ii.safetensors')
image = pipeline('HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. ').images[0]
image.save("my_image.png")

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers