multimodalart/latent-navigation-flux2

Latent Navigation — FLUX.2 Klein

UX-faithful port of LatentNavigation/latentnavigation-flux to FLUX.2 Klein. CLIP-text directions become Qwen3 hidden-state directions; the rest of the experience (1D slider, post-generation slider over a strip of intermediate images, looping video) is preserved.

How directions work

For each (concept_1, concept_2) pair, sample ~64 templated prompt variations a {medium} of a {concept} {subject} and average the per-token Qwen3 hidden- state delta concept_2 − concept_1. At inference, add scale × Δ to the base prompt's embeddings before they enter the transformer.

This is CLIPSlider from Linoy's semantic-sliders repo, translated to Qwen3's stacked hidden states (layers 9/18/27, shape (B, 512, 3·hidden)).

Notes

• —The Space runtime supplies torch, gradio, spaces, huggingface_hub.
• —HF_TOKEN is set as a Space secret (the FLUX.2 Klein model is gated).
• —2D directions are out of scope for v1 — add a second direction state and scale slider; the math composes additively.

Roadmap (image interpolation, separately)

Latent-space slerp on VAE reference tokens did not produce usable interpolations — VAE midpoints sit off the natural-image manifold and the reference path treats them as conditioning gibberish. Three approaches with a real chance to ship later:

1. 1.Flow inversion (RF-Inversion / FlowEdit): invert each image to noise via the model's velocity field, slerp the noise, denoise. Principled answer for rectified flow.
2. 2.Native multi-reference: pass both images as ref tokens (Klein supports this) and steer the mix textually.
3. 3.Image-as-slider-endpoint: encode both images, take the latent delta, apply as an additive scale on a chosen reference — slider semantics on images instead of t∈[0,1] interpolation.