alxcarln/codons

Fungal coding sequence dataset

Dataset of codon usage for fungal organisms created from the Ensembl Genomes clustered to 50% sequence identity at the protein level and split into 80%/10%/10% train/validation/test splits for use in training a neural network to design native-looking nucleotide sequences for fungal organisms

Dataset processing

This document describes the preparation of the fungal codons dataset.

Obtaining the raw data

The raw data, CDS sequences for fungal organisms, was obtained from Ensembl Genomes via the following URL

https://ftp.ensemblgenomes.ebi.ac.uk/pub/fungi/release-57/fasta/

All files were considered, and those matching the pattern "*.cds.all.fa.gz" were downloaded with wget using the following command

wget -r -np -nH -A "*.cds.all.fa.gz" \
    ftp://ftp.ensemblgenomes.ebi.ac.uk/pub/fungi/release-57/fasta/

This results in a dataset of 775,642 nucleotide sequences from 1,506 individual species represented in Ensembl Genomes.

Calling ORFs from the nucleotide sequences

For this step, we keep sequences that start with ATG and are an even multiple of 3 with no ambiguous nucleotides. Also we remove sequences that would result in a protein longer than 512 residues.

Clustering at the protein level

Clustering was performed with MMseqs2 using commands like the following.

mmseqs createdb protein.fa proteinDB
mmseqs cluster -c 0.80 --min-seq-id 0.5 proteinDB clustDB tmp
mmseqs createsubdb clustDB proteinDB repDB
mmseqs convert2fasta repDB rep.fa   

This produces 259,737 clusters at 50% identity (80% coverage for both sequences)

Train/test splits

The dataset was split into 80% training examples (around 200k), 10% validation examples (around 20k), and 10% testing (around 20k) examples