4Fever4/siwa-kershef-sdxl-lora

Siwa Kershef Architecture — SDXL LoRA (style-isolation experiment)

A LoRA for SDXL 1.0 that teaches the earthen kershef architecture of Siwa Oasis, Egypt, behind the trigger token `k3rshef`, evaluated specifically for style bleeding into other architectural styles.

Dataset (112 curated, hand-captioned images, per-image licence): 4Fever4/siwa-kershef-architecture

Usage

import torch
from diffusers import StableDiffusionXLPipeline
pipe = StableDiffusionXLPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda")
pipe.load_lora_weights("4Fever4/siwa-kershef-sdxl-lora", weight_name="siwa_kershef_sdxl_lora_v1.safetensors", adapter_name="siwa")
pipe.set_adapters(["siwa"], adapter_weights=[0.6])   # recommended strength, see evaluation
img = pipe("k3rshef, exterior of a two-storey house, projecting palm-trunk beams, small window openings, restored, photo",
           num_inference_steps=30, guidance_scale=6.5).images[0]

Prompt vocabulary the LoRA was captioned with (usable at inference): palm-trunk beams, thick tapered columns, small window openings, external staircase, crenellated parapet, tapered minaret, covered passage; condition: restored, partially restored, newly built, eroded ruin.

Training

Exact command: train.sh, dataset config: dataset.toml (kohya-ss/sd-scripts).

Evaluation — does the style stay behind its trigger?

scripts/evaluate.py generates the same prompts with the same seeds with base SDXL and with the LoRA loaded.

• —Target prompts (with k3rshef) — the LoRA should pull these toward Siwa.
• —Control prompts (no trigger, LoRA still loaded): Al-Qasr Dakhla (a neighbouring earthen style — the hardest case), Nubian Aswan, modern New Cairo, Mamluk Cairo, Scandinavian — the LoRA should not change these.

Metrics (CLIP ViT-L/14, cosine):

• —siwa_shift = similarity to the training-set centroid (LoRA) − same (base). Target ↑, control ≈ 0.
• —base_vs_lora_sim = similarity between the base image and the LoRA image for the same prompt+seed. Control ≈ 1 means untouched.

Summary

Per prompt (siwa_shift / base-vs-LoRA similarity)

[image] Rows: base SDXL / v1 @1.0 / v1 @0.6. Columns: house + mosque (target, with trigger), Nubian, modern New Cairo, Mamluk (controls, no trigger). Same seed.

What happened — three iterations

1. 1.v1 @1.0 learned the style strongly (target +0.058) but bled into neighbouring Egyptian prompts (control +0.044): modern New Cairo turned into a beige earthen block, Nubian lost its blue paint and composition, Mamluk domes and carving flattened into mud walls. Scandinavian and — notably — Al-Qasr (0.96 similarity) stayed stable, so the leak is semantic ("Egypt / desert → kershef"), not global.
2. 2.v2 (prior-preservation): retrained with 48 base-SDXL regularisation images of neighbouring styles (Nubian, modern Cairo, Mamluk, Ottoman, Mediterranean, riad, desert resort …, captioned without the trigger, wording/seeds disjoint from the eval prompts). Bleeding dropped to +0.009, but the target style collapsed too (+0.007) — the regulariser was too strong for a 112-image set at 2,000 steps. Raising v2 to 1.5 made things worse. Negative result, kept for transparency.
3. 3.v1 @0.6 — recommended. Keeps 84 % of the target shift (+0.049) while cutting control bleeding by 61 % (+0.044 → +0.017); controls stay 0.945 similar to base (Mamluk 0.964, Nordic 0.964, Al-Qasr 0.969). Remaining weak spot: modern New Cairo still drifts (+0.056).

What I would do next

• —Weaker prior preservation (fewer reg repeats / prior_loss_weight 0.3–0.5) instead of 1:1, then re-sweep scale.
• —Add negative-pair captions for the modern case (e.g. modern glass building reg images specifically).
• —More intact-building data (the dataset is still 45 % ruins) and a human rating pass alongside CLIP.
• —Port the same data / captions / evaluation to a Flux LoRA on a 24 GB GPU.

Limitations

• —One-day experiment; 112 images, 3 seeds per prompt — the numbers are indicative, not a benchmark.
• —SDXL, not Flux: Flux LoRA training does not fit an 8 GB GPU. The same data/captions/evaluation pipeline transfers directly to Flux (e.g. ai-toolkit) on a 24 GB GPU.
• —CLIP similarity is a proxy; the grids in eval/ should be looked at, not only the numbers.