abka03/stylsteer-vlm

StyleSteer-VLM — Interactive Activation Steering Demo

Steer VLM captions in real time using training-free activation-steering methods.

Upload an image → pick a style → adjust intensity → see how the model's output changes.

Features

• —3 Backbone VLMs: LLaVA-1.5-7B, Qwen2.5-VL-3B, Gemma-3-4B-IT
• —6 Steering Methods: PromptBase, DiffMean, PCADir, LAT, LinearProbe, SNMF-Diff
• —20 Styles across 4 tracks (sentiment, stylistic, personality, rhetorical)
• —Per-style intensity sliders with linked budget constraint
• —Side-by-side comparison: baseline vs steered captions
• —Auto-resteer: debounced slider changes trigger automatic re-generation

How It Works

The demo applies activation steering — modifying the model's internal hidden-state representations at inference time — to shift output style without any fine-tuning. Steering vectors are pre-computed offline and applied as additive perturbations to selected transformer layers.

Technical Details

• —Frontend: React 18 (served as static files)
• —Backend: FastAPI on port 7860
• —GPU: Required for model inference (CUDA)

Paper

NeurIPS 2025 submission — StyleSteer-VLM: Training-Free Activation Steering for Stylistic Control in Vision-Language Models