andyx10/dit-loras-interpreting
andyx10/dit-loras-interpreting Experimenting with interpreting write vectors over 100 hidden-topic model organisms fromdiff-interpretation-tuning/loras implementation We use 'self_attn.o_proj andmlp.down_proj` for write vectors: two per block across 36 blocks, with a total of 72 write vectors per organism. Jacobian Lens from Neuronpedia neuronpedia/jacobian-lens (qwen3-4b/jlens/Salesforce-wikitext/Qwen3-4B_jacobian_lens.pt) layout test100/… See the full description on the dataset page: https://huggingface.co/datasets/andyx10/dit-loras-interpreting.
andyx10/dit-loras-interpreting
Experimenting with interpreting write vectors over 100 hidden-topic model organisms from`diff-interpretation-tuning/loras`
implementation
We use 'selfattn.oproj andmlp.down_proj` for write vectors: two per block across 36 blocks, with a total of 72 write vectors per organism.
Jacobian Lens from Neuronpedia `neuronpedia/jacobian-lens` (qwen3-4b/jlens/Salesforce-wikitext/Qwen3-4B_jacobian_lens.pt)
layout
test100/ the 100-organism cohort (supersedes the 20)
NN/ 00..99
vectors.pt the 72 write vectors
jlens_readouts.json logit-lens and J-lens top tokens, both signs
lens_scores.pt per-vector x sign x topic identification scores
organism.json topic, trigger, source shard and index
identification.{json,csv}
judge_view.csv exactly what the LLM judge saw, plus its score
llm_judge.json LLM-judge results, both modes, both lenses
llm_judge_detail.csv per-organism guesses and reasoning
jlens_report.md human-readable sheet, all 100
jlens_tokens.csv
random-control/ matched-norm noise null
NN/ the earlier 20-organism run, 00..19
+ readouts.json, generations.jsonl, activation_diagnostics.json
lens_validation.json
dit_baseline.json config for the 20
dit_test100.json config for the 100