hammamiomar/sdxl-turbo-sae-labels

SDXL-Turbo SAE Feature Labels

20,480 labeled sparse autoencoder features across 4 UNet attention blocks in SDXL-Turbo, plus 50K generated images with full activation logs.

Built for latent-dance — a real-time audio-reactive music visualizer using SAE steering at 50 FPS.

Important attribution: The SDXL-Turbo sparse autoencoders/checkpoints used here were trained and released by Surkov et al. / EPFL through `sdxl-unbox`. This dataset does not claim authorship of the SAE training. It contributes labels, activation logs, VLM consensus annotations, factor groupings, and analysis for those upstream SAE features, built for latent-dance. The public checkpoint files are available from the upstream Hugging Face model repository: surokpro2/sdxl-saes.

Upstream SAE Architecture

The SAE architecture and checkpoints come from Surkov et al. / EPFL's `sdxl-unbox` work. latent-dance uses those upstream SAEs and provides the feature labeling and activation-analysis dataset below.

Steered Blocks

Dataset Contents

Labels (sae_labels/)

Images & Activations (sae_images/)

Label Schema

Each entry in final/{block}.json:

{
  "block": "down.2.1",
  "feature_id": 0,
  "label": "office organization supplies",
  "category": "object",
  "confidence": "high",
  "method": "vlm_consensus_3way",
  "mean_activation": 19.95,
  "activation_cv": 0.5575,
  "vlm_labels": {
    "kimi_k2_5": "office organization supplies",
    "qwen3_vl_235b": "office supplies and document organization",
    "glm_4_6v": "Office folders and bind"
  },
  "tfidf_top5": ["holder", "folder", "white", "rack", "open"],
  "spatial_pattern": null,
  "n_activating_images": 0
}

Confidence Levels

Categories

27 categories across all blocks: object, scene, composition, character, face, body, action, style, mood, color, lighting, texture, pattern, material, spatial, symmetry, border, depth, density, contrast, shape, edge, subject, accessory, object_detail, setting, unclear, unknown.

Factor Schema

Each entry in factors/{block}.json:

{
  "factor_id": 0,
  "label": "cute cartoon characters",
  "category": "unknown",
  "n_constituents": 3,
  "constituents": [
    {"feature_id": 4781, "weight": 118.1, "label": "...", "category": "..."},
    {"feature_id": 3745, "weight": 2.56, "label": "cute cartoon characters", "category": "character"}
  ],
  "mean_activation": 0.3151,
  "max_activation": 0.6467,
  "n_active_images": 49735
}

Activation Schema

Each line in activations.jsonl:

{
  "id": 0,
  "prompt": "a photo of a sunset over mountains",
  "down.2.1": [
    {"f": 6, "mean": 20.98, "max": 43.07, "sum": 272.72, "row": 3, "col": 8},
    ...
  ],
  "mid.0": [...],
  "up.0.0": [...],
  "up.0.1": [...]
}

Each block entry contains the top-k (k=10) active features per 16x16 patch, with mean/max/sum aggregation and spatial coordinates.

Pipeline Overview

Total cost: ~$85

Usage

import json
from pathlib import Path

# Load final labels for a block
with open("sae_labels/final/down.2.1.json") as f:
    features = json.load(f)

# Build a lookup
label_map = {f["feature_id"]: f["label"] for f in features}
print(label_map[2301])  # "monstrous fantasy creatures"

# Filter by category
objects = [f for f in features if f["category"] == "object"]
print(f"{len(objects)} object features")

# Load NMF factors
with open("sae_labels/factors/down.2.1.json") as f:
    factors = json.load(f)

for factor in factors[:5]:
    labels = [c["label"] for c in factor["constituents"][:3]]
    print(f"Factor {factor['factor_id']}: {factor['label']} — {labels}")

Acknowledgments

• —SDXL-Turbo SAE architecture and checkpoints by sdxl-unbox / sdxl-unbox.epfl.ch (Surkov et al., EPFL). These SAEs were not trained by latent-dance.
• —NMF factor grouping inspired by Paint With Ember (Goodfire, 2025)
• —VLM annotations via OpenRouter (Qwen3-VL-235B, GLM-4.6V, Kimi K2.5)