datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
uniref50_processed
UniRef50 (Processed, ESM-valid as Validation)
Dataset Summary
This dataset is a preprocessed UniRef50 snapshot tailored for unsupervised protein representation learning. It:
Normalizes sequences (uppercase, * removed), filters by length and ambiguity, and deduplicates by MD5.
Splits by UniRef50 cluster ID to prevent leakage.
Uses the official ESM validation headers as the entire valid split (no sampling).
Provides JSONL.zst shards for efficient streaming with 🤗… See the full description on the dataset page: https://huggingface.co/datasets/DeepFoldProtein/uniref50_processed.twin-uniref50-faiss
Twin-Model UniRef50 FAISS Index
FAISS index over Twin model mean-pooled embeddings of all UniRef50
representative proteins (~49.8M). The Twin model is a two-tower contrastive
encoder fine-tuned on Resnik GO similarity:
Custom tower: AA-vocab Transformer → padding-masked mean pool → MLP → 512-dim
ESM tower: facebook/esm2_t33_650M_UR50D (frozen) → masked mean pool → MLP → 512-dim
Output: concat(custom, esm) → 1024-dim
Checkpoint:… See the full description on the dataset page: https://huggingface.co/datasets/genomenet/twin-uniref50-faiss.cluster_reps_bfd_and_uniref_novelty_pruned_familiesplm_2500_unirefesm2-uniref50-faiss
ESM2 UniRef50 FAISS Index
FAISS index over ESM2 (esm2_t33_650M_UR50D) mean-pooled embeddings of
GO-annotated UniRef50 proteins. Used by the
genomenet/functional-distance
Space for nearest-neighbor search.
Files
File
Description
esm2_uniref50.index
FAISS index (OPQ + IVF + PQ, cosine / inner product on L2-normalized vectors)
ids.npy
UniRef50 cluster IDs aligned with FAISS positions (dtype='S24')
metadata.json
Build parameters (dim, factory, nprobe, n_vectors… See the full description on the dataset page: https://huggingface.co/datasets/genomenet/esm2-uniref50-faiss.