juiceb0xc0de/vibethinker-1.5b-atlas

VibeThinker-1.5B Brain Atlas

This is an internal-mechanics atlas for the 1.5B parameter VibeThinker model. The goal was not to benchmark end-task accuracy, but to map what the network is actually doing with its parameters: where it computes, where it stores behaviorally relevant structure, and which late-layer directions are safe to touch.

What was run

• —Activation census over 9,523 prompts spanning compliance, reasoning, code, math, multilingual, and refusal-style questions.
• —Per-layer feature taxonomy for mlp, gate, up, and attention heads.
• —OV-circuit spectral analysis per head (W_V @ W_O).
• —Sub-Zero surgery pass on every layer, with a capability fence across code, math, reasoning, factual, and multilingual domains.
• —Pipeline was run on a CPU-only environment.

Key geometry

What the numbers suggest

The model is not a lookup table

OV-circuit spectral concentration averages 0.049, with effective rank around 55. That is a distributed signature, not a sparse “copy-paste” attention pattern. Attention heads appear to be doing weighted computation across many directions, not memorizing specific token-to-token jumps.

Feature activation is broad

The feature taxonomy is dominated by partial_shared and broadly_shared classes, with a smaller non_activated tail and very few all_shared features. Most dimensions responded to many prompts rather than one hyper-specific trigger.

Late layers are load-bearing

Sub-Zero finds structured singular-value subspace only in layers 18–27, which is 36% of the network depth. The first half of the model looks like wide preprocessing; the second half does the structured transformation.

Surgical fragility is the main caveat

The capability fence keeps about 74.5% of tested axes, but the rejected ones hit hard:

• —Layer 18 up_proj axis 0 does 0.81 damage to code generation.
• —Layer 18 up_proj axis 0 also scores the highest math, reasoning, and multilingual damage.
• —Several down_proj and gate_proj axes in the early sacred layers fail the fence.

Interpretation: the 1.5B late-layer subspace is doing a lot of work per direction. It has less redundancy than the larger variant, so removing a top singular value tends to break more than one capability at once.

Classifier stability dips in the middle

Sub-Zero classifier accuracy drops to 0.75–0.83 around layers 13–17, then recovers in the late sacred layers. That mid-network region is messier or more entangled than the clean late-layer representation.

Bottom line

VibeThinker-1.5B behaves like a compact reasoning model: distributed attention, broad-feature MLPs, and a deep-but-narrow sacred region where a small number of directions carry most of the task load. It is interpretable, but not easy to edit safely because its late layers are not highly redundant.