bioscan-ml/BarcodeBERT

1---2language:3- en4license: mit5pipeline_tag: feature-extraction6library_name: transformers7description: "Pretrained weights for BarcodeBERT, a transformer for insect DNA barcode inference."8---9 10# BarcodeBERT for Taxonomic Classification11 12A pre-trained transformer model for inference on insect DNA barcoding data, as presented in the paper [BarcodeBERT: Transformers for Biodiversity Analysis](https://huggingface.co/papers/2311.02401).13 14Code: https://github.com/bioscan-ml/BarcodeBERT15 16[Colab](https://colab.research.google.com/drive/1MUEQVHIOX2ks7tLsMoQtNlbvsbSuYgs1)17 18To use **BarcodeBERT** as a feature extractor:19 20```python21from transformers import AutoTokenizer, BertForTokenClassification22 23# Load the tokenizer24tokenizer = AutoTokenizer.from_pretrained(25    "bioscan-ml/BarcodeBERT", trust_remote_code=True26)27 28# Load the model29model = BertForTokenClassification.from_pretrained("bioscan-ml/BarcodeBERT", trust_remote_code=True)30 31# Sample sequence32dna_seq = "ACGCGCTGACGCATCAGCATACGA"33 34# Tokenize35input_seq = tokenizer(dna_seq, return_tensors="pt")["input_ids"]36 37# Pass through the model38output = model(input_seq.unsqueeze(0))["hidden_states"][-1]39 40# Compute Global Average Pooling 41features = output.mean(1)42```43 44## Citation 45 46If you find BarcodeBERT useful in your research please consider citing:47 48    @misc{arias2023barcodebert,49      title={{BarcodeBERT}: Transformers for Biodiversity Analysis},50      author={Pablo Millan Arias51        and Niousha Sadjadi52        and Monireh Safari53        and ZeMing Gong54        and Austin T. Wang55        and Scott C. Lowe56        and Joakim Bruslund Haurum57        and Iuliia Zarubiieva58        and Dirk Steinke59        and Lila Kari60        and Angel X. Chang61        and Graham W. Taylor62      },63      year={2023},64      eprint={2311.02401},65      archivePrefix={arXiv},66      primaryClass={cs.LG},67      doi={10.48550/arxiv.2311.02401},68    }