l20
Datasets
All datasets matching “l20”qwen3-8b-activations-l20-l36
Qwen3 8B Activations for Layers 20 and 36
This dataset contains assistant-token residual activations harvested from Qwen/Qwen3-8B over 980000 training conversations from lmsys/lmsys-chat-1m.
We only generated for Layer 20 and 36 because each one costs 2TB and we simply cannot afford to store more :)
You can use this dataset to train SAEs, linear probes, other mech interp models etc, for Qwen3 8B.
We picked Qwen3 8B because this is a small part of a larger experiment to use feature… See the full description on the dataset page: https://huggingface.co/datasets/sammyliu/qwen3-8b-activations-l20-l36.droid-depth-l20-fullCLIP-ViT-L-14-336-L20-features
OpenAI/CLIP-ViT-L/14@336 Layer 20 features, CLIP+BLIP labels
Feature activation max visualization of the 4096 Features @ L20
CLIP+BLIP labels (may or may not describe what a neuron truly encodes!)
⚠️ May contain sensitive images, albeit abstract. Use responsibly!
Examples:
qwen2.5-7b-instruct-nla-L20-finefineweb-100k
Qwen2.5-7B-Instruct NLA training data — residual stream, block 20
Training data for a Natural Language Autoencoder on Qwen/Qwen2.5-7B-Instruct:
residual-stream activations paired with natural-language explanations of the text
they were taken from.
Unlike the dataset this is derived from, the activation_vector column is
included — every EasyNLA/nanoNLA trainer requires it.
Trained models: https://huggingface.co/Yooniel/qwen2.5-7b-instruct-nla-L20
(AV val ppl 4.07, AR held-out FVE… See the full description on the dataset page: https://huggingface.co/datasets/Yooniel/qwen2.5-7b-instruct-nla-L20-finefineweb-100k.L20latent-activations-gemma-2-9b-it-SimPO-L20-k100-lr1e-04-CCLoss