AlekseyCalvin/Alexander_BLOK_Flux_LoRA_SilverAgePoets_v3

Alexander Blok Flux Low-Rank Adapter (LoRA) for SilverAgePoets.com Version 3 (aka "2_1")

An adapter to reproduce the likeness of the legendary Symbolist/Modernist Russian and Soviet poet: <br> Alexander Blok (b.1880-d.1921) <br>

CLICK HERE TO READ OUR TRANSLATION OF BLOK'S "STRANGER"/"NEZNAKOMKA"

This version of our Blok LoRA was the product of an experimental training to transfer face/attribute-features from historical photos with minimal compute time by using a high rank training, and a relatively high learning rate, but with a minimal number of steps. <br> This is the version at rank128 (linear_dims+alpha), lr of 0.0005, batch size 2 (with a dataset of only 12 images, but x3 resolutions: 512, 768, 1024), minimalist descriptive captions with a dropout of .09 (9%), adamw8bit optimizer, and only 50 steps (!), with no warmups. <br>

All in all, we consider the experiment fairly successful, and exceptionally demonstrative of the unprecedentedly absorbent learning capacity not just among FLUX models, but DiT-based models more broadly. <br> We will soon reproduce this experiment on one of the homebrew de-distilled versions of FLUX, and see whether fast learning improves or diminishes without the extra steering from distilled guidance during fine-tuning. <br> And we are most curious about whether the re-introduction of distilled guidance during inference still zeroes in even on low-step learning. <br> Higher step learning with de-distilled Flux models has so far demonstated broader potentials to any other technique. <br> But that's neither here nor there, as this particular LoRA was trained on a regular distilled version anyhow. <br>

Regarding file size: <br>

Ideally, we ought to extract only the learned features into a much smaller-sized LoRA file. <br> We will get around to doing that eventually. <br> For now, anyone interested in using this locally, please forgive for us the huge file size! <br>

Regarding the weird trigger word: <br>

We have learned about a Flux model knowledge-activation prompting technique of approximating digital camera file names in prompts. <br> This technique (prompting with tokens like "object02.cr2", etc) demonstrably results in more natural "raw" looking photorealistic outputs from the Flux base model(s). So, we decided to run a parallel co-experiment to see whether and how this base knowledge might affect fine-tune training. <br> So, instead of using the poet's name, we simply used a camera file name-like token of 'BLOK02.CR2'. <br>

<Gallery />

Trigger words

You should use BLOK_02.CR2 to trigger the image generation.

Use it with the 🧨 diffusers library

from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained('black-forest-labs/FLUX.1-dev', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('AlekseyCalvin/BlokFlux2_1', weight_name='lora.safetensors')
image = pipeline('your prompt').images[0]

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers