leandrodevai/medical-meadow-medical-flashcards-splits

Dataset Card for Medical Meadow Medical Flashcards - Fixed Splits

Dataset Summary

This dataset is a reproducible train/validation/test split of `flwrlabs/medical-meadow-medical-flashcards`, an English medical question-answering dataset from the MedAlpaca project.

The source dataset contains 33,955 flashcards in a single training split. This version preserves the original rows and columns while assigning every example to one of three fixed splits using seed 42. No medical review, filtering, deduplication, or rewriting was performed as part of the split operation.

Supported Tasks

The dataset is intended for research and experimentation involving:

• —generative medical question answering;
• —instruction tuning and supervised fine-tuning;
• —model selection using a fixed validation set;
• —final evaluation using a held-out test set.

It is not a clinical benchmark and should not be used by itself to establish medical correctness, safety, or readiness for patient-facing use.

Languages

The dataset is in English (en).

Dataset Structure

Data Instances

Each row contains one instruction, one medical question, and its reference answer:

{
  "instruction": "Answer this question truthfully",
  "input": "<medical question>",
  "output": "<reference answer>"
}

Data Fields

• —instruction (string): instruction describing the question-answering task.
• —input (string): medical flashcard question.
• —output (string): reference answer associated with the question.

Data Splits

Usage

from datasets import load_dataset

dataset = load_dataset(
    "leandrodevai/medical-meadow-medical-flashcards-splits"
)

train_dataset = dataset["train"]
validation_dataset = dataset["validation"]
test_dataset = dataset["test"]

To keep results comparable, avoid selecting prompts, models, checkpoints, or hyperparameters after inspecting performance on test.

Dataset Creation

Source Data

All examples come from `flwrlabs/medical-meadow-medical-flashcards`. That dataset originates from MedAlpaca's Medical Meadow collection and was created for medical question-answering model training.

Split Procedure

The original 33,955-row training split was divided with Hugging Face datasets.Dataset.train_test_split:

from datasets import DatasetDict, load_dataset

SEED = 42
source = load_dataset("flwrlabs/medical-meadow-medical-flashcards")

train, remainder = source["train"].train_test_split(
    train_size=30_000,
    seed=SEED,
).values()

validation, test = remainder.train_test_split(
    test_size=0.5,
    seed=SEED,
).values()

dataset = DatasetDict(
    {
        "train": train,
        "validation": validation,
        "test": test,
    }
)

The split is random and is not stratified by topic, specialty, question type, or any demographic attribute.

Personal and Sensitive Information

The dataset contains medical educational content. No dedicated audit for personal, sensitive, or identifying information was performed when creating this split. Users should inspect the data and apply safeguards appropriate to their jurisdiction and use case.

Considerations for Using the Data

Limitations and Biases

• —Reference answers may be incomplete, outdated, ambiguous, or medically incorrect; they were not revalidated by clinicians for this release.
• —No duplicate or near-duplicate removal was performed. Closely related or paraphrased examples may occur within a split or across different splits, which can inflate evaluation results.
• —The random split does not guarantee balanced coverage of medical specialties, diseases, populations, or question difficulty.
• —The dataset is English-only and may reflect the terminology, assumptions, and educational priorities of its source material.
• —A model trained on these flashcards may produce confident but unsafe or unsupported medical statements.

Recommendations

Use this dataset as a research and educational resource. Before any high-stakes, clinical, or patient-facing application, add independent medical expert review, factuality and safety evaluation, demographic bias analysis, privacy checks, and validation on representative external data.

The dataset and models trained on it are not a substitute for professional medical advice, diagnosis, or treatment.

Licensing Information

The upstream Hugging Face dataset declares the Creative Commons license family (cc) but does not identify a specific Creative Commons license version or variant. This split adds no new license to the underlying data. Users are responsible for reviewing the upstream dataset and MedAlpaca repository and for confirming that their intended use complies with all applicable terms.

Citation Information

When using this dataset, cite the original MedAlpaca work and identify this fixed-split derivative in your data statement:

@article{han2023medalpaca,
  title   = {MedAlpaca--An Open-Source Collection of Medical Conversational AI Models and Training Data},
  author  = {Han, Tianyu and Adams, Lisa C. and Papaioannou, Jens-Michalis and
             Grundmann, Paul and Oberhauser, Tom and L{\"o}ser, Alexander and
             Truhn, Daniel and Bressem, Keno K.},
  journal = {arXiv preprint arXiv:2304.08247},
  year    = {2023}
}

Suggested dataset acknowledgement:

We use leandrodevai/medical-meadow-medical-flashcards-splits, a seed-42 train/validation/test split of flwrlabs/medical-meadow-medical-flashcards containing 30,000/1,977/1,978 examples, respectively.

Dataset Card Contact

For questions about the fixed splits, open an issue in the associated project repository. For questions about the source data and its original construction, consult the upstream dataset card and the MedAlpaca repository.