barryhpr/LLM_Modularity_attribution

Per-task neuron attribution scores for Modular Cognitive Architecture Emerges in Large Language Models

Code and analysis pipeline: <https://github.com/Pengrui-Han/LLM_Modularity>

This dataset contains the raw attribution-patching tensors that the GitHub release omits (they are ~6 GB). With these files you can run every overlap / ablation / statistics script in the repo without re-running attribution patching on a GPU.

Layout

results/<model>/<domain>/<task>/
    neuron_attribution.pt          float32 tensor, shape [num_layers, intermediate_size]
    sorted_indices_positive.npy    int array, shape [n_positive, 2]  (layer, unit), best first
    attribution_meta.json          {"total_units", "num_layers", "num_units"}
    baselines.json                 clean / corrupted baselines, both-correct accuracy, example indices

<model> is the HuggingFace id with / and . replaced by _ and - (e.g. Qwen_Qwen2-5-32B-Instruct). <domain> is one of Lan, MD, ToM, phys.

Models: Qwen2.5-32B-Instruct, Qwen2.5-72B-Instruct, OLMo-2-0325-32B-Instruct, Llama-3.1-70B-Instruct, Mistral-Large-Instruct-2407, Mistral-Small-24B-Instruct-2501 (the six models in the paper's main analysis).

Only tasks that passed the 60% both-correct inclusion filter have a neuron_attribution.pt for a given model, so the task set differs slightly per model (35–46 tasks). The task set is exactly the one in the GitHub release (every task that has an attribution_meta.json there).

What the numbers are

For each MLP neuron i (the input to mlp.down_proj, i.e. the post-activation hidden of size intermediate_size), evaluated at the final prompt token with full-sequence teacher forcing:

attribution_i = (clean_act_i − corrupted_act_i) · ∂ metric / ∂ act_i

where the gradient is taken on the corrupted-prompt forward pass and the metric is the normalized log-probability of the correct answer (1 = clean baseline, 0 = corrupted baseline). Positive values mean that restoring the clean activation moves the model toward its clean behaviour. Scores are summed over the both-correct examples of a task.

Selecting a task circuit

The paper uses the top 0.1% positively attributed neurons:

import torch, numpy as np
attr = torch.load("results/<model>/<domain>/<task>/neuron_attribution.pt").numpy()
L, U = attr.shape
k = max(1, int(L * U * 0.1 / 100))          # 0.1 % ; use 1.0 for 1 %
idx = np.load("results/<model>/<domain>/<task>/sorted_indices_positive.npy")[:k]  # (layer, unit) pairs

Equivalently, flatten attr, keep entries > 0, sort descending, take the first k. sorted_indices_positive.npy is just that ordering precomputed.

Download

from huggingface_hub import snapshot_download
snapshot_download("barryhpr/LLM_Modularity_attribution", repo_type="dataset",
                  local_dir="LLM_Modularity")        # drops files into results/…

Or a single model: add allow_patterns=["results/allenai_OLMo-2-0325-32B-Instruct/**"].

Citation

See the GitHub repository; citation will be added upon publication.