aayu-outdooragi/latentlens-repro

LatentLens reproduction bundle

Independent reproduction of LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs (Krojer et al., ICML 2026 — arXiv:2602.00462, OpenReview qJFatzEGQc).

Interactive Three.js visualization: Latent Space Observatory (see ui/latentlens_ui.html).

What's here

• —code/core.py — model-agnostic implementation of LatentLens, LogitLens, EmbeddingLens + the contextualized-representation index and the mid-layer-leap analysis.
• —code/run_qwen2vl.py — the GPU experiment (Qwen2-VL-7B): builds the contextual index, extracts visual-token latents, computes the mid-layer leap (Claim 3) and qualitative outputs (Claim 4).
• —code/judge_job.py — the VLM-judge interpretability eval (Claim 1), Qwen2.5-VL-72B via Inference Providers.
• —code/smoke_test.py — local CPU smoke test (gpt2).
• —qwen2vl_7b/ — results.json, midlayerleap.npz, qualitative.json, lensoutputs.json.
• —judge/ — judge_results.json.
• —figures/ — mid-layer-leap heatmap, leap curve, Claim-1 bars (PNG + HTML + CSV).
• —ui/ — the self-contained interactive visualization.

Headline results (off-the-shelf Qwen2-VL-7B-Instruct)

• —Claim 3 (Mid-Layer Leap) ✅ — early visual layers align to text layers 2–4, never to the embedding layer (0.0%); same-layer diagonal emerges at mid/late layers.
• —Claim 4 ✅ — LatentLens → real words (shoes, leather, cage, dogs); LogitLens → subword noise.
• —Claim 1 ✅ (ordering) — LatentLens 60.0% ≫ EmbeddingLens 5.0% > LogitLens 1.7%; 60% matches the paper's own 55–62% for off-the-shelf Qwen2-VL-7B.
• —Claims 2/5/6 documented (training grid / linear-connector ablation / human study out of scope).

Corpus = 15,000 COCO dense captions (documented substitute for Visual Genome region descriptions). Full method, caveats, and Job URLs in the Trackio logbook.