InstaDeepAI/ChatNT

1---2library_name: transformers3pipeline_tag: text-generation4---5 6# ChatNT7 8[ChatNT](https://www.biorxiv.org/content/10.1101/2024.04.30.591835v1) is the first multimodal conversational agent designed with a deep understanding of biological sequences (DNA, RNA, proteins). 9It enables users — even those with no coding background — to interact with biological data through natural language and it generalizes across multiple biological tasks and modalities.10 11**Developed by:** [InstaDeep](https://huggingface.co/InstaDeepAI)12 13### Model Sources14 15<!-- Provide the basic links for the model. -->16 17- **Repository:** [Nucleotide Transformer](https://github.com/instadeepai/nucleotide-transformer)18- **Paper:** [ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein Tasks](https://www.biorxiv.org/content/10.1101/2024.04.30.591835v1.full.pdf) 19 20 21### License Summary221. The Licensed Models are **only** available under this License for Non-Commercial Purposes.232. You are permitted to reproduce, publish, share and adapt the Output generated by the Licensed Model only for Non-Commercial Purposes and in accordance with this License.243. You may **not** use the Licensed Models or any of its Outputs in connection with:25    1. any Commercial Purposes, unless agreed by Us under a separate licence;26    2. to train, improve or otherwise influence the functionality or performance of any other third-party derivative model that is commercial or intended for a Commercial Purpose and is similar to the Licensed Models;27    3. to create models distilled or derived from the Outputs of the Licensed Models, unless such models are for Non-Commercial Purposes and open-sourced under the same license as the Licensed Models; or28    4. in violation of any applicable laws and regulations.29 30### Architecture and Parameters  31ChatNT is built on a three‑module design: a 500M‑parameter [Nucleotide Transformer v2](https://www.nature.com/articles/s41592-024-02523-z) DNA encoder pre‑trained on genomes from 850 species 32(handling up to 12 kb per sequence, Dalla‑Torre et al., 2024), an English‑aware Perceiver Resampler that linearly projects and gated‑attention compresses 332048 DNA‑token embeddings into 64 task‑conditioned vectors (REF), and a frozen 7B‑parameter [Vicuna‑7B](https://lmsys.org/blog/2023-03-30-vicuna/) decoder.34 35Users provide a natural‑language prompt containing one or more `<DNA>` placeholders and the corresponding DNA sequences (tokenized as 6‑mers). 36The projection layer inserts 64 resampled DNA embeddings at each placeholder, and the Vicuna decoder generates free‑form English responses in 37an autoregressive fashion, using low‑temperature sampling to produce classification labels, multi‑label statements, or numeric values.38 39### Training Data  40ChatNT was instruction‑tuned on a unified corpus covering 27 diverse tasks from DNA, RNA and proteins, spanning multiple species, tissues and biological processes. 41This amounted to 605 million DNA tokens (≈ 3.6 billion bases) and 273 million English tokens, sampled uniformly over tasks for 2 billion instruction tokens.42Examples of questions and sequences for each task, as well as additional task information, can be found in [Datasets_overview.csv](Datasets_overview.csv).43 44### Tokenization  45DNA inputs are broken into overlapping 6‑mer tokens and padded or truncated to 2048 tokens (~ 12 kb). English prompts and 46outputs use the LLaMA tokenizer, augmented with `<DNA>` as a special token to mark sequence insertion points.47 48### Limitations and Disclaimer  49ChatNT can only handle questions related to the 27 tasks it has been trained on, including the same format of DNA sequences. ChatNT is **not** a clinical or diagnostic tool.50It can produce incorrect or “hallucinated” answers, particularly on out‑of‑distribution inputs, and its numeric predictions may suffer digit‑level errors. Confidence 51estimates require post‑hoc calibration. Users should always validate critical outputs against experiments or specialized bioinformatics 52pipelines.53 54### Other notes55We also provide the params for the ChatNT jax model in `jax_params`.56 57## How to use58 59Until its next release, the transformers library needs to be installed from source with the following command in order to use the models. 60PyTorch should also be installed.61 62```63pip install --upgrade git+https://github.com/huggingface/transformers.git64pip install torch sentencepiece65```66 67A small snippet of code is given here in order to **generate ChatNT answers from a pipeline (high-level)**.68- The prompt used for training ChatNT is already incorporated inside the pipeline and is the following:69  "A chat between a curious user and an artificial intelligence assistant that can handle bio sequences. The assistant gives helpful,70  detailed, and polite answers to the user's questions."71 72```73# Load pipeline74from transformers import pipeline75pipe = pipeline(model="InstaDeepAI/ChatNT", trust_remote_code=True)76 77# Define custom inputs (note that the number of <DNA> token in the english sequence must be equal to len(dna_sequences))78english_sequence = "Is there any evidence of an acceptor splice site in this sequence <DNA> ?"79dna_sequences = ["ATCGGAAAAAGATCCAGAAAGTTATACCAGGCCAATGGGAATCACCTATTACGTGGATAATAGCGATAGTATGTTACCTATAAATTTAACTACGTGGATATCAGGCAGTTACGTTACCAGTCAAGGAGCACCCAAAACTGTCCAGCAACAAGTTAATTTACCCATGAAGATGTACTGCAAGCCTTGCCAACCAGTTAAAGTAGCTACTCATAAGGTAATAAACAGTAATATCGACTTTTTATCCATTTTGATAATTGATTTATAACAGTCTATAACTGATCGCTCTACATAATCTCTATCAGATTACTATTGACACAAACAGAAACCCCGTTAATTTGTATGATATATTTCCCGGTAAGCTTCGATTTTTAATCCTATCGTGACAATTTGGAATGTAACTTATTTCGTATAGGATAAACTAATTTACACGTTTGAATTCCTAGAATATGGAGAATCTAAAGGTCCTGGCAATGCCATCGGCTTTCAATATTATAATGGACCAAAAGTTACTCTATTAGCTTCCAAAACTTCGCGTGAGTACATTAGAACAGAAGAATAACCTTCAATATCGAGAGAGTTACTATCACTAACTATCCTATG"]80 81# Generate sequence82generated_english_sequence = pipe(83  inputs={84    "english_sequence": english_sequence,85    "dna_sequences": dna_sequences86  }87)88 89# Expected output: "Yes, an acceptor splice site is without question present in the sequence."90```91 92A small snippet of code is given here in order to **infer with the model without any abstraction (low-level)**.93 94```95import numpy as np96from transformers import AutoModel, AutoTokenizer97 98# Load model and tokenizers99model = AutoModel.from_pretrained("InstaDeepAI/ChatNT", trust_remote_code=True)100english_tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/ChatNT", subfolder="english_tokenizer")101bio_tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/ChatNT", subfolder="bio_tokenizer")102 103# Define custom inputs (note that the number of <DNA> token in the english sequence must be equal to len(dna_sequences))104# Here the english sequence should include the prompt105english_sequence = "A chat between a curious user and an artificial intelligence assistant that can handle bio sequences. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Is there any evidence of an acceptor splice site in this sequence <DNA> ?"106dna_sequences = ["ATCGGAAAAAGATCCAGAAAGTTATACCAGGCCAATGGGAATCACCTATTACGTGGATAATAGCGATAGTATGTTACCTATAAATTTAACTACGTGGATATCAGGCAGTTACGTTACCAGTCAAGGAGCACCCAAAACTGTCCAGCAACAAGTTAATTTACCCATGAAGATGTACTGCAAGCCTTGCCAACCAGTTAAAGTAGCTACTCATAAGGTAATAAACAGTAATATCGACTTTTTATCCATTTTGATAATTGATTTATAACAGTCTATAACTGATCGCTCTACATAATCTCTATCAGATTACTATTGACACAAACAGAAACCCCGTTAATTTGTATGATATATTTCCCGGTAAGCTTCGATTTTTAATCCTATCGTGACAATTTGGAATGTAACTTATTTCGTATAGGATAAACTAATTTACACGTTTGAATTCCTAGAATATGGAGAATCTAAAGGTCCTGGCAATGCCATCGGCTTTCAATATTATAATGGACCAAAAGTTACTCTATTAGCTTCCAAAACTTCGCGTGAGTACATTAGAACAGAAGAATAACCTTCAATATCGAGAGAGTTACTATCACTAACTATCCTATG"]107 108# Tokenize109english_tokens = english_tokenizer(english_sequence, return_tensors="pt", padding="max_length", truncation=True, max_length=512).input_ids110bio_tokens = bio_tokenizer(dna_sequences, return_tensors="pt", padding="max_length", max_length=512, truncation=True).input_ids.unsqueeze(0) # unsqueeze to simulate batch_size = 1111 112# Predict113outs = model(114    multi_omics_tokens_ids=(english_tokens, bio_tokens),115    projection_english_tokens_ids=english_tokens,116    projected_bio_embeddings=None,117)118 119# Expected output: Dictionary of logits and projected_bio_embeddings120```