datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
agentclinic_medqa
AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
Release
[05/18/2024] 🤗 We added support for HuggingFace models!
[05/17/2024] We release new results and support for GPT-4o!
[05/13/2024] 🔥 We release AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environment. We propose a multimodal benchmark based on language agents which simulate the clinical environment. Checkout the paper and the… See the full description on the dataset page: https://huggingface.co/datasets/katielink/agentclinic_medqa.MedQAbstain
🩺 MedQAbstain
MedQAbstain is a benchmark for evaluating medical abstention in Large Language Models (LLMs), introduced in the paper:
LLMs (Almost) Never Abstain Under Medical Uncertainty
It repurposes standard medical multiple-choice QA (MCQA) datasets to test whether models can refrain from unsafe actions when no clinically appropriate choice is available.
🔗 Paper: LLMs (Almost) Never Abstain Under Medical Uncertainty🔗 GitHub repository:… See the full description on the dataset page: https://huggingface.co/datasets/disi-unibo-nlp/MedQAbstain.MedQA_ODD_Hindi_testMedQA_tran_COT_Bengali_test2Multimodal_MedQA
Multilingual-Multimodal-MedQA (M3-MedQA)
This dataset is a multilingual and multimodal medical VQA benchmark extended from the WorldMedQA-V dataset. It evaluates the medical reasoning performance and cross-lingual consistency of VLMs across five languages: English, Korean, Japanese, Arabic, and Wolof.
📂 Dataset Structure
Images: Provided in .parquet format for efficient loading and high accessibility.
Metadata: A comprehensive .csv file containing questions, options… See the full description on the dataset page: https://huggingface.co/datasets/pnu-clink/Multimodal_MedQA.MedQA_ODD_Punjabi_testMedQA_ODD_Bengali_testMedQA_ODD_Assamese_testMedQA_ODD_Tamil_testMedQA_ODD_Marathi_testMedQA_ODD_Telugu_testMedQA_tran_COT_Telugu_train2MedQA_tran_COT_Assamese_trainMedQA_tran_COT_English_testMedQA_tran_COT_Bengali_train2MedQA_tran_COT_Assamese_train2MedQA_tran_COT_Telugu_test2