richardyoung/CardioEmbed-BioLinkBERT
035
CardioEmbed-BioLinkBERT
Domain-specialized cardiology text embeddings using LoRA-adapted BioLinkBERT-large
This is the best performing model from our comparative study of 10 embedding architectures for clinical cardiology.
Performance
Usage
from transformers import AutoModel, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModel.from_pretrained("michiyasunaga/BioLinkBERT-large")
tokenizer = AutoTokenizer.from_pretrained("michiyasunaga/BioLinkBERT-large")
# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "richardyoung/CardioEmbed-BioLinkBERT")
# Generate embeddings
text = "Atrial fibrillation with rapid ventricular response"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
outputs = model(**inputs)
embeddings = outputs.last_hidden_state.mean(dim=1)Training
- Training Data: 106,535 cardiology text pairs from medical textbooks
- Method: LoRA fine-tuning (r=16, alpha=32)
- Loss: Multiple Negatives Ranking Loss (InfoNCE)
Citation
@article{young2024comparative,
title={Comparative Analysis of LoRA-Adapted Embedding Models for Clinical Cardiology Text Representation},
author={Young, Richard J and Matthews, Alice M},
journal={arXiv preprint},
year={2024}
}Related Models
This is part of the CardioEmbed model family. See richardyoung/CardioEmbed for more models.