datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
uniref50-test
Dataset Card for UniRef50
Dataset Summary
[More Information Needed]
Supported Tasks and Leaderboards
[More Information Needed]
Languages
[More Information Needed]
Dataset Structure
Data Instances
[More Information Needed]
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation
Curation Rationale
[More Information Needed]
Source Data… See the full description on the dataset page: https://huggingface.co/datasets/zgcarvalho/uniref50-test.uniref50esm2_uniref_pretraining_data
ESM-2 Uniref Pretraining Data
Dataset Description:
UniRef, or UniProt Reference Clusters, are databases of clustered protein sequences from the UniProt Knowledgebase (UniProtKB) that group similar sequences to reduce redundancy and make data easier to work with for biological research. It offers different levels of clustering (UniRef100, UniRef90, and UniRef50) based on sequence identity, with each cluster containing a representative sequence, a count of member proteins… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/esm2_uniref_pretraining_data.uniref90
UniRef90
Complete UniRef90 dataset from UniProt, converted from XML to sharded Parquet. UniRef90 clusters sequences at 90% identity, providing a non-redundant protein sequence resource that balances comprehensiveness with reduced redundancy.
Part of the ConvergeBio Protein Database Collection — see also UniRef100, UniRef50, and UniClust30.
Dataset Summary
Clusters
188,848,220
Shards
386
Compressed size
~52 GB (zstd)
Sequence lengths
11 – 49,499 aa… See the full description on the dataset page: https://huggingface.co/datasets/ConvergeBio/uniref90.uniref-50-foldseek-v1uniref50uniref50
Dataset Card for UniRef50
UniRef50 data downloaded from https://www.uniprot.org/help/downloads on January 24, 2024.
Uniref50_fulluniref50uniref100
Dataset Card for UniRef100
Dataset Summary
UniRef100 data downloaded on January 24, 2024.
uniref90UniRef50_512_alluniref50-2025-10-v4uniref50
UniRef50
Complete UniRef50 dataset from UniProt, converted from XML to sharded Parquet. UniRef50 clusters sequences at 50% identity, providing the most aggressively deduplicated UniRef tier — ideal for training protein language models and building diverse, non-redundant sequence sets.
Part of the ConvergeBio Protein Database Collection — see also UniRef90, UniRef100, and UniClust30.
Dataset Summary
Clusters
60,315,044
Shards
130
Compressed size
~17 GB… See the full description on the dataset page: https://huggingface.co/datasets/ConvergeBio/uniref50.uniref100
UniRef100
Complete UniRef100 dataset from UniProt, converted from XML to sharded Parquet. UniRef100 contains every unique protein sequence in UniProtKB plus selected UniParc records, providing the most comprehensive non-identical sequence resource available.
Part of the ConvergeBio Protein Database Collection — see also UniRef90, UniRef50, and UniClust30.
Dataset Summary
Clusters
475,217,233
Shards
970
Compressed size
~133 GB (zstd)
Sequence lengths
2 –… See the full description on the dataset page: https://huggingface.co/datasets/ConvergeBio/uniref100.uniref
Dataset Card for flair-bio/uniref
Dataset Summary
This dataset is a cleaned, deduplication-clustered, and quality-scored version of
UniRef100 (UniProt Reference Clusters), a comprehensive, non-redundant-by-design database of
protein sequences derived from UniProtKB and select UniParc records. It has been reprocessed by
the FLAIR modules/data pipeline into a single
training-ready Parquet dataset (sharded), with additional per-sequence redundancy reduction
(MMseqs2… See the full description on the dataset page: https://huggingface.co/datasets/flair-bio/uniref.uniref50-sorted-structure-tokenuniref50uniref50AF2_UniRef50AF2_UniRef50 is organized in LMDB format. If you want to load it, you have to first download it and then open the file using the lmdb package.
Here is an example of how you can get samples:
import lmdb
lmdb_dir = "/your/path/to/AF2_UniRef50/train"
with lmdb.open(lmdb_dir, readonly=True).begin() as txn:
length = int(txn.get('length'.encode()).decode())
for i in range(length):
data_str = txn.get(str(i).encode()).decode()
data = json.loads(data_str)
print(data)… See the full description on the dataset page: https://huggingface.co/datasets/westlake-repl/AF2_UniRef50.nanoplm-uniref50-3M-subset
NanoPLM UniRef50 3M Subset
A 3,000,000-sequence subset of UniRef50 protein
sequences, pre-split into train/validation sets. Sequences are filtered to a length of
20 to 512 amino acids (inclusive). Intended for pretraining and experimenting with
small protein language models (PLMs).
Splits
Split
File
Sequences
train
train.fasta
2,950,200
validation
validation.fasta
49,800
total
3,000,000
Format
The dataset is provided as Parquet… See the full description on the dataset page: https://huggingface.co/datasets/heispv/nanoplm-uniref50-3M-subset.uniref50-replay-mix-v1
uniref50-replay-mix-v1
Stage-1 continued-pretraining corpus for
eshmun-vocab: protein sequences (UniRef50) mixed with
a general/biomedical/math/code text replay slice, so a vocab-extended LLM
(e.g. khairi/qwen3-0.6b-protein-vocab-v0) learns protein-sequence
statistics without catastrophically forgetting its pretrained language
ability. Design rationale and target ratios: see docs/pretrain-dataset-mix.md
in the eshmun-vocab repo.
Objective: plain next-token prediction. Every row… See the full description on the dataset page: https://huggingface.co/datasets/khairi/uniref50-replay-mix-v1.uniref90_parquets_shuffledUniRef90 parquet files created by Jude Wells 2025-03-18
see script data_creation_scripts/shuffling/shuffle_uniref90.sh for shuffling it was done
Processing was done on kaspian computer.
uniref50
This dataset was downloaded from UniProt and uploaded on Huggingface on 14/Dec/2024
Number of examples: 68,360,540
Number of tokens/amino acids: 19,582,371,375
Uniref50
Uniref50: Uniref Sequences clustered at 50% sequence identity
~40M Protein Sequences.
Split into train val and test.
Usage
from datasets import load_dataset
# Step 1: Load the dataset from HuggingFace Hub
dataset = load_dataset("zhangzhi/Uniref50")
# Step 2: Access a specific split (e.g., "train", "validation", "test")
train_split = dataset["train"]
print(f"Number of sequences in the train split: {len(train_split)}")
Uniref50_Tokenized_maxlen1024uniref50uniref50_processed
UniRef50 (Processed, ESM-valid as Validation)
Dataset Summary
This dataset is a preprocessed UniRef50 snapshot tailored for unsupervised protein representation learning. It:
Normalizes sequences (uppercase, * removed), filters by length and ambiguity, and deduplicates by MD5.
Splits by UniRef50 cluster ID to prevent leakage.
Uses the official ESM validation headers as the entire valid split (no sampling).
Provides JSONL.zst shards for efficient streaming with 🤗… See the full description on the dataset page: https://huggingface.co/datasets/DeepFoldProtein/uniref50_processed.uniref50-2025-10
UniRef50 Dataset for Protein Language Model Training
This dataset is derived from UniRef50 (Release 2025 October) and prepared for protein language model pre-training.
Dataset Statistics
Original UniRef50 sequences: 58,875,981
Synthetic constructs removed: 439
Validation set size: 294,377 (0.5%)
Initial train set size: 58,581,165
Sequences removed by similarity filter (≥50% identity): 457,727
Final train set size: 58,123,438
Total sequences in dataset: 58,417,815… See the full description on the dataset page: https://huggingface.co/datasets/alejoacelas/uniref50-2025-10.uniref50-sorted-foldseek