microsoft/Dayhoff-MMseqs2

Dayhoff FASTA and MMseqs2 databases

This dataset contains the original Dayhoff Atlas GigaRef and UniRef50 datasets, in formats amenable to MMSeqs2 CPU and GPU utilities.

The train, validation, and test sets from the original atlas were combined and the following datasets available:

• —GigaRef No Singletons - The GigaRef dataset, with no singleton clusters.
• —GigaRef Singletons - The GigaRef dataset, with only singleton clusters.
• —GigaRef Full - Every sequence contained in both the no-singletons and singletons subsets.
• —UniRef50 - UniProt clustered at 50% sequence identity.

Each dataset is or will be available in the following formats:

• —FASTA - Canonical sequence storage format, usable with many bioinformatics tools.
• —MMSeqs2-CPU - Converted folder of unindexed database files compatible with MMSeqs2-CPU. Searches can be tuned to splits that accommodate your system RAM.
• —MMSeqs2-GPU - Converted folder of padded sequence databases for MMSeqs2-GPU search. Requires 1+ GPU(s) on your machine to run.

Current Repo Organization

fastas/
├── gigaref-full.fasta.gz
├── gigaref-singletons.fasta.gz
├── gigaref-no-singletons.fasta.gz
└── uniref50.fasta.gz

mmseqs-cpu/
├── gigaref-singletons/db/
├── gigaref-no-singletons/db/
└── uniref50/db/

mmseqs-gpu/
├── gigaref-singletons/db_gpu/
├── gigaref-no-singletons/db_gpu/
└── uniref50/db_gpu/

MMseqs can read the .fasta.gz files directly.

Put the extracted MMseqs database and temporary search directory on the fastest local SSD or NVMe available. I/O speed, RAM, and GPUs improve throughput; slow storage substantially increases search time.

Hugging Face download example

export REPO=microsoft/Dayhoff-MMseqs2
export REV=main
export DEST=/data/Dayhoff-MMseqs2
export TARGET=gigaref-no-singletons

# FASTA
hf download "$REPO" "fastas/$TARGET.fasta.gz" \
  --repo-type dataset --revision "$REV" --local-dir "$DEST"

# CPU MMseqs
hf download "$REPO" --repo-type dataset --revision "$REV" \
  --include "mmseqs-cpu/$TARGET/**" --local-dir "$DEST"

# GPU MMseqs
hf download "$REPO" --repo-type dataset --revision "$REV" \
  --include "mmseqs-gpu/$TARGET/**" --local-dir "$DEST"

Valid FASTA targets are gigaref-full, gigaref-singletons, gigaref-no-singletons, and uniref50. CPU and GPU MMseqs targets are gigaref-singletons, gigaref-no-singletons, and uniref50.

Extract and search

The FASTA needs no extraction for MMseqs. To create an uncompressed FASTA:

pigz -dc "fastas/$TARGET.fasta.gz" > "fastas/$TARGET.fasta"

Extract either MMseqs representation once:

find "mmseqs-cpu/$TARGET" -type f -name '*.gz' -print0 |
  xargs -0 -n1 pigz -d

find "mmseqs-gpu/$TARGET" -type f -name '*.gz' -print0 |
  xargs -0 -n1 pigz -d

The resulting target prefixes are:

mmseqs-cpu/$TARGET/db/db
mmseqs-gpu/$TARGET/db_gpu/db_gpu

For a 64 GB host, use native MMseqs target splitting:

mmseqs search queryDB TARGET_DB resultDB tmp \
  --gpu 1 \
  --split-mode 0 \
  --split-memory-limit 48G \

Omit --gpu 1 for CPU search.

The UniRef50 and singleton databases were built directly from their published combined FASTAs, so their sequence identifiers match. The no-singletons FASTA and MMseqs database contain the same 1.8B-sequence corpus, but the FASTA uses gr_<source-index> identifiers while the existing MMseqs database retains older g<part>_<row> identifiers. This difference does not affect inference.