CoolFace
Modelpublic

Manhph2211/Q-HEART

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
1likes70downloads
Model Card

<div align="center" style="font-size: 1.5em;"> <strong>Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs (ECAI 2025)</strong> </div>

<div align="center"> <a href="https://github.com/manhph2211/Q-HEART/"><img src="https://img.shields.io/badge/Website-QHEART WebPage-blue?style=for-the-badge"></a> <a href="https://arxiv.org/pdf/2505.06296"><img src="https://img.shields.io/badge/arxiv-Paper-red?style=for-the-badge"></a> <a href="https://huggingface.co/Manhph2211/Q-HEART"><img src="https://img.shields.io/badge/Checkpoint-%F0%9F%A4%97%20Hugging%20Face-White?style=for-the-badge"></a> </div>

<div align="center"> <a href="https://github.com/manhph2211/" target="blank">Hung&nbsp;Manh&nbsp;Pham</a> &emsp; <a href="" target="blank">Jialu&nbsp;Tang</a> &emsp; <a href="https://aqibsaeed.github.io/" target="blank">Aaqib&nbsp;Saeed</a> &emsp; <a href="https://www.dongma.info/" target="blank">Dong&nbsp;Ma</a> &emsp; </div> <br>

Usage

After we have access to meta-llama/Llama-3.2-1B-Instruct model and install suitable transformers package version, we can run:

python
# transformers==4.43.3 accelerate==1.0.1 peft==0.13.2
from transformers import AutoModel
model = AutoModel.from_pretrained("Manhph2211/Q-HEART", trust_remote_code=True, dtype="auto")

Or

bash
git clone https://github.com/manhph2211/Q-HEART.git && cd Q-HEART
conda create -n qheart python=3.9
conda activate qheart
pip install torch --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

Download the checkpoint from here and place it at ckpts/pytorch_model.bin, then run evaluation:

bash
python main.py --model_type meta-llama/Llama-3.2-1B-Instruct --mapping_type Transformer

Citation

bibtex

@article{pham2025q,
  title={Q-Heart: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
  author={Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
  journal={arXiv preprint arXiv:2505.06296},
  year={2025}
}

@inproceedings{pham2025qheart,
  title     = {Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
  author    = {Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
  booktitle = {Proceedings of the European Conference on Artificial Intelligence (ECAI)},
  series    = {Frontiers in Artificial Intelligence and Applications},
  volume    = {413},
  pages     = {4545--4552},
  year      = {2025},
  publisher = {IOS Press},
  doi       = {10.3233/FAIA251356}
}

<div align="center"> Please refer to our <a href="https://github.com/manhph2211/Q-HEART">GitHub repo</a> for more details! </div>