Layered-Labs/benchbase-medmcqa

<h1 align="center">BenchBase: MedMCQA</h1>

<h3 align="center">193,000+ multiple-choice medical questions from Indian postgraduate medical entrance exams, spanning 2,400 healthcare topics.</h3>

<p align="center"> <a href="https://huggingface.co/datasets/Layered-Labs/benchbase-medmcqa"> <img src="https://huggingface.co/datasets/huggingface/badges/resolve/main/dataset-on-hf-md.svg" alt="Dataset on HuggingFace"/> </a> &nbsp; <img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"/> </p>

<p align="center"> <img src="https://huggingface.co/datasets/Layered-Labs/benchbase-medmcqa/resolve/main/banner.svg" alt="BenchBase: MedMCQA" width="100%"/> </p>

Overview

BenchBase: MedMCQA is a repackaging of the MedMCQA dataset for the Layered Labs BenchBase benchmark suite. The source dataset contains 193,155 multiple-choice questions drawn from AIIMS and PGMR medical entrance examinations, covering 2,400+ healthcare topics across clinical medicine, pharmacology, pathology, anatomy, biochemistry, and more. Each question includes four answer options, a correct answer label, subject and topic annotations, and optional expert explanations. The train split (182,822 questions) includes correct answer labels; the test split (6,150 questions) withholds them for evaluation. This repackaging standardizes column names and encoding to match the BenchBase schema.

Statement of Need

Evaluating medical language models requires standardized, clinically relevant benchmarks. MedMCQA is one of the largest and most topic-diverse medical QA datasets available, but the original release uses inconsistent column names and encoding that complicate integration into multi-benchmark evaluation pipelines. This BenchBase repackaging normalizes the schema, documents all fields explicitly, and makes the dataset drop-in compatible with other BenchBase medical benchmarks for unified evaluation.

Intended Use

This dataset is intended for researchers and engineers evaluating or fine-tuning language models on medical question answering. It is appropriate for benchmarking general medical knowledge, subject-specific clinical reasoning, and model calibration across 2,400+ topics. It is not intended as a clinical decision support tool or as a substitute for professional medical judgment.

Limitations

Questions are drawn from Indian postgraduate medical entrance exams and may reflect regional clinical guidelines, drug names, and diagnostic conventions that differ from US or European practice. The test split withholds correct answer labels, so evaluation requires submission to the original leaderboard for held-out accuracy. Expert explanations are available for only a subset of training questions. Some questions have ambiguous or disputed correct answers as noted in the research literature.

Dataset Structure

Splits

Features

Usage

from datasets import load_dataset

ds = load_dataset("Layered-Labs/benchbase-medmcqa")
print(ds)

Example

from datasets import load_dataset

ds = load_dataset("Layered-Labs/benchbase-medmcqa")

# Inspect the training split
train = ds["train"]
print(train[0])

# Filter to pharmacology questions
pharma = train.filter(lambda x: x["subject"] == "Pharmacology")
print(f"Pharmacology questions: {len(pharma)}")

Citation

If you use this dataset in your research, please cite:

@dataset{layeredlabs_benchbase_medmcqa,
  title     = {BenchBase: MedMCQA},
  author    = {Ridwan, Abdullah and Hossain, Radhyyah},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/Layered-Labs/benchbase-medmcqa}
}

License

Released under the MIT License. Maintained by Layered Labs.