CoolFace
Datasetpublic

anonymous2509/TempoMed-Bench-MCQ

TempoMed-Bench-MCQ Data Overview TempoMed-Bench-MCQ is a multiple-choice question benchmark designed to evaluate temporal awareness in medical large language models. Each instance is constructed from a pair of medical guidelines: an up-to-date guideline and an oudated guideline. The question asks about the recommendation according to the up-to-date guideline, while the answer choices include the up-to-date recommendation, the outdated recommendation, plausible… See the full description on the dataset page: https://huggingface.co/datasets/anonymous2509/TempoMed-Bench-MCQ.

sourceHugging Facecc-by-nc-nd-4.0updated 5mo agoView on Hugging Face
0likes12downloads
Dataset Card

TempoMed-Bench-MCQ

Data Overview

TempoMed-Bench-MCQ is a multiple-choice question benchmark designed to evaluate temporal awareness in medical large language models. Each instance is constructed from a pair of medical guidelines: an up-to-date guideline and an oudated guideline. The question asks about the recommendation according to the up-to-date guideline, while the answer choices include the up-to-date recommendation, the outdated recommendation, plausible distractors, and an “I do not know” option.

The dataset contains 721 rows in a single train split and is stored in JSONL format. Each row includes PubMed identifiers and publication years for both the up-to-date and outdated guidelines, a clinical question, five answer choices, the correct answer label, and an explanation.

Schema

Each record contains the following fields:

FieldTypeDescription
idxintegerUnique index of the MCQ instance.
PMID_currentstringPubMed identifier of the up-to-date guideline.
Year_currentintegerPublication year of the up-to-date guideline.
PMID_priorstringPubMed identifier of the outdated guideline.
Year_priorintegerPublication year of the outdated guideline.
QuestionstringClinical question or scenario asking about the recommendation under the up-to-date guideline.
Choice_AstringAnswer choice A.
Choice_BstringAnswer choice B.
Choice_CstringAnswer choice C.
Choice_DstringAnswer choice D.
Choice_EstringAnswer choice E, usually “I do not know the answer.”
CorrectstringCorrect answer label.
ExplanationstringExplanation of the correct answer and its relation to the up-to-date/outdated guideline.

Example Use Cases

This dataset can be used to:

  • —evaluate whether LLMs can identify up-to-date medical recommendations;
  • —test whether models confuse up-to-date recommendations with outdated ones;
  • —study temporal awareness in medical knowledge;
  • —benchmark medical LLMs under guideline changes over time;
  • —analyze model robustness in distinguishing guideline versions.