universitytehran/PerMed-MM

PerMed-MM: A Multimodal, Multi-Specialty Persian Medical Benchmark

**🤗 Dataset** | **📖 Paper** | **📄 PDF**

Dataset Description

PerMed-MM is a multimodal, multi-specialty benchmark designed to evaluate Vision Language Models (VLMs) on Persian medical question answering.

The dataset consists of 733 multiple-choice questions sourced from the Iranian National Medical Board Exams (years 2021 and 2023). Each question is paired with 1 to 5 clinically relevant images, totaling 944 images across 46 medical specialties and multiple visual modalities.

Dataset Statistics

Image Modality Distribution

Images per Question

• —1 image: 82.3% (603 questions)
• —2 images: 11.3% (83 questions)
• —3 images: 2.0% (15 questions)
• —4 images: 4.1% (30 questions)
• —5 images: 0.3% (2 questions)

Citation

If you use this dataset or find it helpful in your research, please cite our paper:

@inproceedings{khoramfar-etal-2025-permed,
    title = "{P}er{M}ed-{MM}: A Multimodal, Multi-Specialty {P}ersian Medical Benchmark for Evaluating Vision Language Models",
    author = "Khoramfar, Ali  and Dousti, Mohammad Javad  and Faili, Heshaam",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.ijcnlp-short.21/",
    doi = "10.18653/v1/2025.ijcnlp-short.21",
    pages = "232--241",
    ISBN = "979-8-89176-299-2"
}