Datasetpublic

danieez/Med-Evidence-2.6k

Med-Evidence-2.6k Med-Evidence-2.6k is a benchmark for evaluating evidence-grounded medical diagnostic reasoning. It was developed as part of the work EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents, accepted to Findings of EMNLP 2026. The benchmark provides clinical diagnostic questions paired with ground-truth answers and annotated evidence spans, enabling evaluation of both diagnostic accuracy and evidence-grounded reasoning. Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/danieez/Med-Evidence-2.6k.

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

1likes86downloads

Dataset Card

Med-Evidence-2.6k

Med-Evidence-2.6k is a benchmark for evaluating evidence-grounded medical diagnostic reasoning. It was developed as part of the work EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents, accepted to Findings of EMNLP 2026.

The benchmark provides clinical diagnostic questions paired with ground-truth answers and annotated evidence spans, enabling evaluation of both diagnostic accuracy and evidence-grounded reasoning.

Dataset Structure

Med-Evidence-2.6k currently contains three subsets:

—DiagnosisArena
—JAMA
—MedXpertQA-Text

Note: During dataset curation, we selected 234 diagnosis-related cases from MedXpertQA-Text and annotated evidence for this subset.

Each subset is provided as a separate Hugging Face configuration.

python

from datasets import load_dataset

dataset = load_dataset(
    "danieez/Med-Evidence-2.6k",
    "DiagnosisArena"
)

print(dataset["test"][0])

The other subsets can be loaded by replacing "DiagnosisArena" with "JAMA" or "MedXpertQA-Text".

Data Format

Each example follows a unified schema:

json

{
  "original_id": "Text-107",
  "dataset": "MedXpertQA-Text",
  "ground_truth_text": "Truncus arteriosus",
  "question_text": "A male neonate, aged 2 weeks, ...",
  "golden_evidence": [
    {
      "original_text": "a single S2",
      "type": "Inclusion"
    },
    {
      "original_text": "all peripheral pulses are bounding",
      "type": "Differentiation"
    }
  ]
}

Fields

Field	Description
`original_id`	Identifier of the example in the original source dataset.
`dataset`	Name of the source subset in Med-Evidence-2.6k.
`ground_truth_text`	Ground-truth diagnosis or answer.
`question_text`	Full clinical question or case description presented to the model.
`golden_evidence`	Annotated evidence spans relevant to the ground-truth answer.
`golden_evidence.original_text`	Evidence span extracted directly from the question text.
`golden_evidence.type`	Functional type of the evidence in diagnostic reasoning.

Intended Use

Med-Evidence-2.6k is intended for research on:

—medical diagnostic reasoning;
—evidence identification and extraction;
—evidence-grounded question answering;
—evaluation of clinical reasoning in large language models.

The dataset supports separate or joint evaluation of diagnostic answer correctness and evidence selection.