datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
sae-activations-llama-3.1-8b-layer19-lmsys-chat-1m
SAE Feature Activations — Llama 3.1 8B Instruct, Layer 19 (LMSYS-Chat-1M)
This dataset contains Sparse Autoencoder (SAE) feature activations extracted from layer 19 of Meta's Llama 3.1 8B Instruct on conversations from LMSYS-Chat-1M.
It also has natural language explainations of features generated by GPT OSS 120B. See subset 4 for details.
The SAE used is Goodfire/Llama-3.1-8B-Instruct-SAE-l19, which decomposes layer-19 residual stream activations into interpretable sparse features.… See the full description on the dataset page: https://huggingface.co/datasets/scaleinvariant/sae-activations-llama-3.1-8b-layer19-lmsys-chat-1m.llama8b-layer15-llamascope-500
Llama8B LlamaScope SAE Steering
This repository contains SAE steering data accompanying the paper Learning a Generative Meta-Model of LLM Activations.
Project page: https://generative-latent-prior.github.io
Code: https://github.com/g-luo/generative_latent_prior
Quick Start
With this data, you can reproduce the SAE steering experiment (Section 4.1, Figure 5) on Llama-3.1-8B.
The 500 concepts are random Layer 15 LlamaScope L15R-32x features with Neuronpedia… See the full description on the dataset page: https://huggingface.co/datasets/generative-latent-prior/llama8b-layer15-llamascope-500.llama8b-layer15-meta-neurons
Llama8B Meta-Neurons
This repository contains meta-neuron data accompanying the paper Learning a Generative Meta-Model of LLM Activations.
Project page: https://generative-latent-prior.github.io
Code: https://github.com/g-luo/generative_latent_prior
Quick Start
With this data, you can browse the 98304 meta-neurons of the Llama-3.1-8B GLP (glp-llama8b-d6, Layer 15).
Meta-neurons are the post-SwiGLU activations of the GLP's MLP blocks. For each meta-neuron… See the full description on the dataset page: https://huggingface.co/datasets/generative-latent-prior/llama8b-layer15-meta-neurons.iclr_kernel_steering_self_awareness_general_llama3.2-1B-it_layer10_activations