datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
gaia-dr3-oa-neuron-xp-spectra
Gaia DR3 OA neuron XP spectra
This table contains the prototype BP/RP spectrum attached to each neuron in the 30 × 30 self-organising map produced by Gaia's Apsis Outlier Analysis module. Measured from the served table, every (neuron_id, xp_spectrum_prototype_wavelength) pair is unique across its 78,300 rows. Each of the 900 neurons has the same ordered grid of 87 wavelengths from 379.0935 to 1034.2460 nm. Neuron-level statistics and other attributes belong to Gaia's separate… See the full description on the dataset page: https://huggingface.co/datasets/astro-legacy-archive/gaia-dr3-oa-neuron-xp-spectra.neuronpedia-sae-concepts
Neuronpedia SAE Concepts
Complete extraction of all individual concepts from every Sparse Autoencoder (SAE) released on Neuronpedia, plus all public features from Anthropic's Towards Monosemanticity (2023) and Scaling Monosemanticity (2024) papers.
Quick Start
from datasets import load_dataset
# Full Neuronpedia dataset (77M rows, streaming recommended)
ds = load_dataset("hbe/neuronpedia-sae-concepts", split="train", streaming=True)
# Unique concepts with essential… See the full description on the dataset page: https://huggingface.co/datasets/hbe/neuronpedia-sae-concepts.pythia-12b-neuron-dataset-examples
pythia-12b-neuron-dataset-examples
This dataset contains the top 64 highest activating dataset examples for each
MLP neuron in Pythia-12b. The dataset examples are all 16 tokens long. See
https://confirmlabs.org/posts/dreaming.html for details.
Columns:
layer: the layer of the neuron
neuron: the index of the neuron
rank: the rank of the example
activation: the activation of the neuron on the example
position: the token position for which the neuron is maximally activated.
text: the… See the full description on the dataset page: https://huggingface.co/datasets/Confirm-Labs/pythia-12b-neuron-dataset-examples.neuronovo-utc-data-glue-mnlicoffee_sales_datallama8b-layer15-meta-neurons
Llama8B Meta-Neurons
This repository contains meta-neuron data accompanying the paper Learning a Generative Meta-Model of LLM Activations.
Project page: https://generative-latent-prior.github.io
Code: https://github.com/g-luo/generative_latent_prior
Quick Start
With this data, you can browse the 98304 meta-neurons of the Llama-3.1-8B GLP (glp-llama8b-d6, Layer 15).
Meta-neurons are the post-SwiGLU activations of the GLP's MLP blocks. For each meta-neuron… See the full description on the dataset page: https://huggingface.co/datasets/generative-latent-prior/llama8b-layer15-meta-neurons.neuronovo-utc-data-goemotionsneuronovo-utc-persent-docneuronovo-utc-data-glue-colaLlama-3.2-3B_neuron-activationsLlama-3.1-8B_neuron-activationsneuronovo-utc-tweeteval-sentimentneuronovo-utc-unhealthy-conversationsneuronovo-utc-measuring-hate-speechargilla_preferences_personalized_filteredrlhf_synthetic_generalizedneuronovo-utc-tweeteval-emotionsneuronovo-utc-hate-speech18-sentencesamortized-neuron-pruning-effects
Amortized Neuron-Ablation Effects (for pruning)
Exact mean-ablation effects of individual MLP neurons in transformer LMs, paired
with cheap forward/backward signals, for the task of amortized causal
intervention-effect prediction at neuron granularity (pruning).
The intervention unit is a single MLP neuron (blocks.{l}.mlp.hook_post channel).
For every (prompt, neuron) we record the exact effect of replacing that neuron's
last-token activation with its dataset mean, m(x… See the full description on the dataset page: https://huggingface.co/datasets/almogtavor/amortized-neuron-pruning-effects.OLMo-7B-0424-hf_neuron-activationsThis dataset contains activation data of neurons in OLMo-7B-0424.
(We define a neuron as a hidden dimension in a MLP sublayer.)
To create the dataset, the model was run on 20M tokens from Dolma (in the same collection, we also release the Dolma subset, which we call dolma-small).
Dataset Description
Each row corresponds to a neuron, identified by the columns "layer" and "neuron".
(We use zero-based indexing).
The other columns are as follows:
The first two elements of the name… See the full description on the dataset page: https://huggingface.co/datasets/sjgerstner/OLMo-7B-0424-hf_neuron-activations.qwen-h-neurons-datasetgemma-2-2b_neuron-activations