CoolFace
Datasetpublic

biohub/ESMC-SAE-Features

ESMC Sparse Autoencoder Features Table This dataset contains a Parquet table of the 16,384 features from the ESMC-6B-sae-layer60-k64-codebook16384, that was used for analysis in the ESMC paper and to construct the ESM Atlas. This table provides descriptions of the precomputed features that can be activated through the spotlight SAE model, assisting users for downstream interpretation of the insights revealed by ESMC. Download the table here. The features descriptions are in the… See the full description on the dataset page: https://huggingface.co/datasets/biohub/ESMC-SAE-Features.

sourceHugging Facemitupdated 4mo agoView on Hugging Face
5likes324downloads
Dataset Card

ESMC Sparse Autoencoder Features Table

This dataset contains a Parquet table of the 16,384 features from the ESMC-6B-sae-layer60-k64-codebook16384, that was used for analysis in the ESMC paper and to construct the ESM Atlas. This table provides descriptions of the precomputed features that can be activated through the spotlight SAE model, assisting users for downstream interpretation of the insights revealed by ESMC.

Download the table here.

The features descriptions are in the table below.

FeatureDescription
feature_idThe feature index.
descriptionFull description of the concept the feature activates on, produced by a multi-agent system based on activations of the feature in Swissprot. This description has 3 sections which are used to derive the summary, activation_pattern, and exemplar_protein_families columns. Note that these represent our best understanding of what each feature represents but are not necessarily comprehensive and may not capture nuances of features that have complex meaning or limited activation in Swissprot.
summarySummary of the biological concept that is activated.
activation_patternThe residue-level activation pattern of the feature with respect to the concepts described in the summary (ex. is the activation diffuse across an entire domain, or is it highly localized to one or more residues in specific locations).
exemplar_protein_familiesSome of the common protein families/functions that the feature is found in. These are derived from the Swissprot data used to generate the description. These should not be considered exhaustive.
thresholdThe activation threshold above which the feature description can be relied upon for accuracy.
categoryAgent-generated categorization of the feature into 1 of 12 categories based on its description. Categories include things like "catalytic function", "compositional bias", and "structural motif".
uniref90_frequencyFrequency of activation of this feature in Uniref90 (total 208M proteins).
uniref90_idfInverse document frequency (IDF) of this feature in Uniref90 (derived from the frequency by log(N / freq), used for feature normalization.
uniref90_max_activationMaximum activation value in Uniref90 (used alongside IDF for feature normalization).
top_100_uniref_idsTop 100 proteins in Uniref90 by activation strength of this feature.
top_swissprot_activationsTop 100 proteins in a non-redundant subset of Swissprot (30% sequence identity clustered and filtered to high annotation score).
decoder_nearest_neighbors10 nearest neighbor features in the SAE decoder latent space, i.e. the most semantically similar features according to the model.