multimolecule/basenji

1---2datasets:3- multimolecule/gencode4library_name: multimolecule5license: agpl-3.06pipeline: regulatory-track7pipeline_tag: other8tags:9- Biology10- DNA11- dna12widget:13- example_title: tumor protein p5314  pipeline_tag: regulatory-track15  sequence_type: DNA16  task: regulatory-track17  text: ACTCCCCTGCCCTCAACAAGATGTTTTGCCAACTGGCCAAGACCTGCCCTGTGCAGCTGTGGGTTGATTCCACACCCCCGCCCGGCACCCGCGTCCGCGCCATGGCCATCTACAAGCAGTCACAGCACATGACGGAGGTTGTGAGGCGCTGCCCCCACCATGAGCGCTGCTCAGATAGCGATGG18- example_title: BRCA1 DNA repair associated19  pipeline_tag: regulatory-track20  sequence_type: DNA21  task: regulatory-track22  text: TCATTGGAACAGAAAGAAATGGATTTATCTGCTCTTCGCGTTGAAGAAGTACAAAATGTCATTAATGCTATGCAGAAAATCTTAGAGTGTCCCATCTGG23- example_title: hemoglobin subunit beta24  pipeline_tag: regulatory-track25  sequence_type: DNA26  task: regulatory-track27  text: CATTTGCTTCTGACACAACTGTGTTCACTAGCAACCTCAAACAGACACCATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGGTGAACGTGGATGAAGTTGGTGGTGAGGCCCTGGGCAGG28- example_title: CF transmembrane conductance regulator29  pipeline_tag: regulatory-track30  sequence_type: DNA31  task: regulatory-track32  text: ACTTCACTTCTAATGGTGATTATGGGAGAACTGGAGCCTTCAGAGGGTAAAATTAAGCACAGTGGAAGAATTTCATTCTGTTCTCAGTTTTCCTGGATTATGCCTGGCACCATTAAAGAAAATATCATCTTTGGTGTTTCCTATGATGAATATAGATACAGAAGCGTCATCAAAGCATGCCAACTAGAAGAG33- example_title: telomerase reverse transcriptase34  pipeline_tag: regulatory-track35  sequence_type: DNA36  task: regulatory-track37  text: CGCGGGGGTGGCCGGGGCCAGGGCTTCCCACGTGCGCAGCAGGACGCAGCGCTGCCTGAAACTCGCGCCGCGAGGAGAGGGCGGGGCCGCGGAAAGGAAGGGGAGGGGCTGGGAGGGCCCGGAGGGGGCTGGGCCGGGGACCCGGGAGGGGTCGGGACGGGGCGGGGTCCGCGCGGAGGAGGCGGAGCTGGAAGGTGAAGGGGCAGGACGGGTGCCCGGGTCCCCAGTCCCTCCGCCACGTGGGAAGCGCGGTCCTGGGCGTCTGTGCCCGCGAATCCACTGGGAGCCCGGCCTGGCCCCGACAGCGCAGCTGCTCCGGGCGGACCCGGGG38- example_title: KRAS proto-oncogene39  pipeline_tag: regulatory-track40  sequence_type: DNA41  task: regulatory-track42  text: GCCTGCTGAAAATGACTGAATATAAACTTGTGGTAGTTGGAGCTGGTGGCGTAGGCAAGAGTGCCTTGACGATACAGCTAATTCAGAATCATTTTGTGGACGAATATGATCCAACAATAGAG43- example_title: prion protein (Kanno blood group)44  pipeline_tag: regulatory-track45  sequence_type: cDNA46  task: regulatory-track47  text: ATGGCGAACCTTGGCTGCTGGATGCTGGTTCTCTTTGTGGCCACATGGAGTGACCTGGGCCTCTGC48- example_title: interleukin 1049  pipeline_tag: regulatory-track50  sequence_type: cDNA51  task: regulatory-track52  text: ATGCACAGCTCAGCACTGCTCTGTTGCCTGGTCCTCCTGACTGGGGTGAGGGCC53- example_title: Zaire ebolavirus54  pipeline_tag: regulatory-track55  sequence_type: cDNA56  task: regulatory-track57  text: AATGTTCAAACACTTTGTGAAGCTCTGTTAGCTGATGGTCTTGCTAAAGCATTTCCTAGCAATATGATGGTAGTCACAGAGCGTGAGCAAAAAGAAAGCTTATTGCATCAAGCATCATGGCACCACACAAGTGATGATTTTGGTGAGCATGCCACAGTTAGAGGGAGTAGCTTTGTAACTGATTTAGAGAAATACAATCTTGCATTTAGATATGAGTTTACAGCACCTTTTATAGAATATTGTAACCGTTGCTATGGTGTTAAGAATGTTTTTAATTGGATGCATTATACAATCCCACAGTGTTAT58- example_title: SARS coronavirus59  pipeline_tag: regulatory-track60  sequence_type: cDNA61  task: regulatory-track62  text: ATGTTTATTTTCTTATTATTTCTTACTCTCACTAGTGGTAGTGACCTTGACCGGTGCACCACTTTTGATGATGTTCAAGCTCCTAATTACACTCAACATACTTCATCTATGAGGGGGGTTTACTATCCTGATGAAATTTTTAGATCAGACACTCTTTATTTAACTCAGGATTTATTTCTTCCATTTTATTCTAATGTTACAGGGTTTCATACTATTAATCATACGTTTGACAACCCTGTCATACCTTTTAAGGATGGTATTTATTTTGCTGCCACAGAGAAATCAAATGTTGTCCGTGGTTGGGTTTTTGGTTCTACCATGAACAACAAGTCACAGTCGGTGATTATTATTAACAATTCTACTAATGTTGTTATACGAGCATGTAACTTTGAATTGTGTGACAACCCTTTCTTTGCTGTTTCTAAACCCATGGGTACACAGACACATACTATGATATTCGATAATGCATTTAAATGCACTTTCGAGTACATATCT63- example_title: insulin64  pipeline_tag: regulatory-track65  sequence_type: cDNA66  task: regulatory-track67  text: ATGGCCCTGTGGATGCGCCTCCTGCCCCTGCTGGCGCTGCTGGCCCTCTGGGGACCTGACCCAGCCGCAGCCTTTGTGAACCAACACCTGTGCGGCTCACACCTGGTGGAAGCTCTCTACCTAGTGTGCGGGGAACGAGGCTTCTTCTACACACCCAAGACCCGCCGGGAGGCAGAGGACCTGCAGGTGGGGCAGGTGGAGCTGGGCGGGGGCCCTGGTGCAGGCAGCCTGCAGCCCTTGGCCCTGGAGGGGTCCCTGCAGAAGCGTGGCATTGTGGAACAATGCTGTACCAGCATCTGCTCCCTCTACCAGCTGGAGAACTACTGCAACTAG68- example_title: cyclin dependent kinase inhibitor 2A69  pipeline_tag: regulatory-track70  sequence_type: cDNA71  task: regulatory-track72  text: ATGGAGCCGGCGGCGGGGAGCAGCATGGAGCCTTCGGCTGACTGGCTGGCCACGGCCGCGGCCCGGGGTCGGGTAGAGGAGGTGCGGGCGCTGCTGGAGGCGGGGGCGCTGCCCAACGCACCGAATAGTTACGGTCGGAGGCCGATCCAGGTCATGATGATGGGCAGCGCCCGAGTGGCGGAGCTGCTGCTGCTCCACGGCGCGGAGCCCAACTGCGCCGACCCCGCCACTCTCACCCGACCCGTGCACGACGCTGCCCGGGAGGGCTTCCTGGACACGCTGGTGGTGCTGCACCGGGCCGGGGCGCGGCTGGACGTGCGCGATGCCTGGGGCCGTCTGCCCGTGGACCTGGCTGAGGAGCTGGGCCATCGCGATGTCGCACGGTACCTGCGCGCGGCTGCGGGGGGCACCAGAGGCAGTAACCATGCCCGCATAGATGCCGCGGAAGGTCCCTCAGACATCCCCGATTGA73- example_title: human papillomavirus type 16 E674  pipeline_tag: regulatory-track75  sequence_type: cDNA76  task: regulatory-track77  text: ATGCACCAAAAGAGAACTGCAATGTTTCAGGACCCACAGGAGCGACCCAGAAAGTTACCACAGTTATGCACAGAGCTGCAAACAACTATACATGATATAATATTAGAATGTGTGTACTGCAAGCAACAGTTACTGCGACGTGAGGTATATGACTTTGCTTTTCGGGATTTATGCATAGTATATAGAGATGGGAATCCATATGCTGTATGTGATAAATGTTTAAAGTTTTATTCTAAAATTAGTGAGTATAGACATTATTGTTATAGTTTGTATGGAACAACATTAGAACAGCAATACAACAAACCGTTGTGTGATTTGTTAATTAGGTGTATTAACTGTCAAAAGCCACTGTGTCCTGAAGAAAAGCAAAGACATCTGGACAAAAAGCAAAGATTCCATAATATAAGGGGTCGGTGGACCGGTCGATGTATGTCTTGTTGCAGATCATCAAGAACACGTAGAGAAACCCAGCTGTAA78---79 80# Basenji81 82Deep convolutional neural network for predicting genomic coverage tracks across chromosomes.83 84## Disclaimer85 86This is an UNOFFICIAL implementation of [Sequential regulatory activity prediction across chromosomes with deep convolutional and recurrent neural networks](https://doi.org/10.1101/gr.227819.117) by David R. Kelley, Yakir A. Reshef, et al.87 88The OFFICIAL repository of Basenji is at [calico/basenji](https://github.com/calico/basenji).89 90> [!TIP]91> The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.92 93**The team releasing Basenji did not write this model card for this model so this model card has been written by the MultiMolecule team.**94 95## Model Details96 97Basenji is a deep convolutional neural network trained to predict genomic regulatory activity from long DNA sequences. It consumes a long DNA window (~131 kb), passes it through a convolution + pooling stem that downsamples the sequence, and then through a tower of dilated residual convolutional blocks that expand the receptive field. A pointwise output head predicts a vector of genomic coverage tracks for each output bin. Because the stem downsamples the input, the prediction is **binned**: the output has shape `(batch_size, num_bins, num_tracks)` where each bin summarizes 128 bp of sequence and `num_tracks` is the number of genomic coverage experiments.98 99### Model Specification100 101| Input Length | Bin Size | Output Bins | Hidden Size | Dilated Blocks | Num Labels | Num Parameters (M) | FLOPs (G) | MACs (G) | Max Num Tokens |102| ------------ | -------- | ----------- | ----------- | -------------- | ---------- | ------------------ | --------- | -------- | -------------- |103| 131,072      | 128      | 896         | 768         | 11             | 5,313      | 30.09              | 234.85    | 117.19   | 131,072        |104 105FLOPs and MACs are measured on the canonical 131,072 bp Basenji input window.106 107### Links108 109- **Code**: [multimolecule.basenji](https://github.com/DLS5-Omics/multimolecule/tree/master/multimolecule/models/basenji)110- **Data**: ENCODE, FANTOM5, GTEx, and related genomic coverage tracks aligned to human and mouse genomes111- **Paper**: [Sequential regulatory activity prediction across chromosomes with deep convolutional and recurrent neural networks](https://doi.org/10.1101/gr.227819.117)112- **Developed by**: David R. Kelley, Yakir A. Reshef, Maxwell Bileschi, David Belanger, Cory Y. McLean, Jasper Snoek113- **Model type**: 1D dilated residual CNN with pre-activation blocks for binned multi-track genomic coverage prediction114- **Original Repository**: [calico/basenji](https://github.com/calico/basenji)115 116## Usage117 118The model file depends on the [`multimolecule`](https://multimolecule.danling.org) library. You can install it using pip:119 120```bash121pip install multimolecule122```123 124### Direct Use125 126#### Genomic Coverage Prediction127 128You can use this model to predict binned genomic coverage tracks from a DNA sequence:129 130```python131>>> import torch132>>> from multimolecule import DnaTokenizer, BasenjiConfig, BasenjiForTokenPrediction133 134>>> config = BasenjiConfig(135...     sequence_length=256, stem_channels=8, conv_tower_channels=[8],136...     stem_pool_size=2, head_hidden_size=8, crop_bins=2, num_labels=4,137...     blocks={"num_blocks": 1, "kernel_size": 3, "bottleneck_size": 4},138... )139>>> model = BasenjiForTokenPrediction(config)140>>> output = model(torch.randint(config.vocab_size, (1, 256)))141>>> output.logits.shape142torch.Size([1, 60, 4])143>>> coverage, channels = model.postprocess(output)144>>> coverage.shape145torch.Size([1, 60, 4])146```147 148The binned positional axis is treated as the "token" axis: each output position corresponds to one149genomic bin rather than a single nucleotide.150 151### Interface152 153- **Input length**: fixed 131,072 bp DNA window154- **Output binning**: 128 bp per output bin; 896 output bins per window (after `Cropping1D(64)` on each side)155- **Output**: raw pre-softplus `logits` of shape `(batch_size, num_bins, num_tracks)`; use `postprocess` for non-negative coverage tracks156 157## Training Details158 159Basenji was trained to predict genomic coverage tracks (DNase-seq, ATAC-seq, ChIP-seq and CAGE) from160the human and mouse reference genomes.161 162### Training Data163 164The model was trained on a large compendium of functional genomics experiments aligned to the human165(hg38) and mouse (mm10) reference genomes. The genome was divided into overlapping windows; for each166window the per-128-bp coverage of every experiment served as the regression target.167 168### Training Procedure169 170#### Pre-training171 172The model was trained to minimize a Poisson regression loss between predicted and observed coverage.173 174## Citation175 176```bibtex177@article{kelley2018sequential,178  author    = {Kelley, David R. and Reshef, Yakir A. and Bileschi, Maxwell and Belanger, David and McLean, Cory Y. and Snoek, Jasper},179  title     = {Sequential regulatory activity prediction across chromosomes with deep convolutional and recurrent neural networks},180  journal   = {Genome Research},181  year      = 2018,182  volume    = 28,183  number    = 5,184  pages     = {739--750},185  doi       = {10.1101/gr.227819.117},186  publisher = {Cold Spring Harbor Laboratory}187}188```189 190> [!NOTE]191> The artifacts distributed in this repository are part of the MultiMolecule project.192> If MultiMolecule supports your research, please cite the MultiMolecule project as follows:193 194```bibtex195@software{chen_2024_12638419,196  author    = {Chen, Zhiyuan and Zhu, Sophia Y.},197  title     = {MultiMolecule},198  doi       = {10.5281/zenodo.12638419},199  publisher = {Zenodo},200  url       = {https://doi.org/10.5281/zenodo.12638419},201  year      = 2024,202  month     = may,203  day       = 4204}205```206 207## Contact208 209Please use GitHub issues of [MultiMolecule](https://github.com/DLS5-Omics/multimolecule/issues) for any questions or comments on the model card.210 211Please contact the authors of the [Basenji paper](https://doi.org/10.1101/gr.227819.117) for questions or comments on the paper/model.212 213## License214 215This model implementation is licensed under the [GNU Affero General Public License](license.md).216 217For additional terms and clarifications, please refer to our [License FAQ](license-faq.md).218 219```spdx220SPDX-License-Identifier: AGPL-3.0-or-later221```