MohamedAhmedAE/Med_LLaMa3_fine-tuning_dataset

Med-LLaMA3 — Medical Instruction Fine-Tuning Dataset

A large, unified medical instruction-tuning corpus (~1.65 million examples) compiled, cleaned, and standardized from a diverse set of public medical sources. It is the training corpus used to fine-tune the Med-LLaMA3 family (LLaMA-3.2 1B/3B and LLaMA-3.1 8B) in the paper “Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models” (Applied Sciences, 2026). All sources were consolidated into a single instruction-style schema spanning medical examinations, clinical dialogues, biomedical literature, and encyclopedic references.

• —📄 Paper: Med-LLaMA3 (Applied Sciences 2026, 16(12), 6158) · DOI: 10.3390/app16126158
• —💻 Code: github.com/Mohamed-Ahmed-Abo-El-Enen/MasterPapers
• —🤖 Models trained on this data: 1B · 3B · 8B (adapters; merged versions also available)

Dataset summary

Supported tasks

• —Medical question answering — open-ended factual/clinical questions and USMLE-style reasoning.
• —Multiple-choice QA — exam-style questions with enumerated options.
• —Clinical dialogue — patient-message → doctor-style response.
• —Biomedical literature tasks — e.g., summarizing an abstract into a title; causal-relationship classification.
• —Clinical case comprehension — reading and understanding structured case passages.

These are delivered in a unified instruction format suitable for supervised fine-tuning (SFT) / instruction-tuning of causal language models.

Dataset structure

Data fields

Each row has six string fields:

Record types (type)

• —`QA` — open-ended questions answered truthfully, doctor-style patient responses, abstract-to-title summarization, and relationship-classification prompts. choices is empty.
• —`MCQ` — multiple-choice questions; choices is populated and output is the correct option.
• —`CASE` — clinical case passages presented for reading/understanding (instruction “Study the following case carefully and understand its details.”); input carries the passage.

Example rows

{
  "instruction": "Answer this question truthfully",
  "input": "What is the role of macrophages in the formation of foam cells?",
  "context": "",
  "choices": "",
  "output": "Macrophages take up excess cholesterol in the subendothelium and become lipid-laden foam cells, an early step in atherosclerotic plaque formation.",
  "type": "QA"
}

{
  "instruction": "Choose the correct answer for the following question",
  "input": "Which part of the nephron normally reabsorbs the most water?",
  "context": "The proximal tubule reabsorbs roughly two-thirds of filtered water...",
  "choices": "1) Proximal tubule. 2) Thick ascending limb. 3) Distal tubule. 4) Collecting duct.",
  "output": "1",
  "type": "MCQ"
}

Splits

How to use

from datasets import load_dataset

ds = load_dataset("MohamedAhmedAE/Med_LLaMa3_fine-tuning_dataset")
print(ds)
print(ds["train"][0])

A simple Alpaca-style prompt formatter:

def format_example(ex):
    parts = [ex["instruction"].strip()]
    if ex.get("context") and ex["context"] not in ("", "None"):
        parts.append(f"\nContext:\n{ex['context'].strip()}")
    if ex.get("input"):
        parts.append(f"\nQuestion:\n{ex['input'].strip()}")
    if ex.get("choices"):
        parts.append(f"\nChoices:\n{ex['choices'].strip()}")
    prompt = "\n".join(parts) + "\n\nAnswer:\n"
    return {"prompt": prompt, "completion": ex["output"].strip()}

train = ds["train"].map(format_example)

Tip: filter by type to build task-specific subsets, e.g. ds["train"].filter(lambda r: r["type"] == "MCQ").

Dataset creation

Curation taxonomy

To make the corpus a reproducible methodological artifact, the paper organizes all sources along three axes: (A) source type (examination QA, clinical dialogue, biomedical literature, encyclopedic reference), (B) clinical granularity (basic science, clinical reasoning, patient communication), and (C) task format (multiple-choice, open-ended QA, generative dialogue). Sources were then mapped into the unified instruction schema above.

Source composition (per the paper, Table 1)

English-language counts as reported in the paper. Exact counts in this repository may differ slightly due to cleaning, deduplication, and the train/validation split.

Preprocessing & quality assurance

As described in the paper, the pipeline included: medical-expert validation of source content; deduplication via semantic-similarity analysis to prevent leakage; terminology standardization using UMLS concepts; ethical/safety filtering to remove potentially harmful advice; and steps toward balanced representation across specialties, difficulty levels, and task types. The corpus-assembly and cleaning scripts are released in the code repository.

Evaluation integrity

The eight MMLU medical subsets (Anatomy, Clinical Knowledge, College Biology, College Medicine, Medical Genetics, Nutrition, Professional Medicine, Virology) were used only for held-out evaluation and were excluded from the fine-tuning corpus. For benchmarks with official splits (MedMCQA, MedQA-USMLE, PubMedQA), only the official training partitions were drawn into this corpus.

Considerations for using the data

Intended use

Research on medical NLP and instruction-tuning of language models. Not for clinical use. Models trained on this data must not be used for autonomous diagnosis, treatment, or any decision affecting patient care without a qualified clinician in the loop.

Limitations & biases

• —English only, and skewed toward exam-style and dialogue formats; long-form, multi-turn, and non-English clinical text are under-represented.
• —Source bias: counts are dominated by a few large sources (clinical notes, encyclopedic references, dialogue), which may bias models toward those styles. Rare conditions may be under-represented.
• —Empty/`"None"` fields: context, choices, and occasionally output can be empty depending on type; handle these in preprocessing.
• —Possible residual noise: despite cleaning, some abstract-summarization rows include non-medical or multilingual abstracts, and some case rows are bibliographic snippets.
• —Content reflects the medical knowledge and any biases present in the underlying sources; verify against authoritative references before any applied use.

Personal & sensitive information

Clinical-note sources (e.g., MIMIC-III) are de-identified at source, but users handling any clinical text should follow applicable privacy regulations (e.g., HIPAA, GDPR) and avoid re-identification attempts.

Licensing & source restrictions

This is an aggregated corpus; the component sources carry their own, differing licenses, and some are access-controlled or non-commercial. Notably:

• —MIMIC-III is distributed under the PhysioNet Credentialed Health Data License and a Data Use Agreement that normally requires credentialed access — review your obligations before redistributing or building on this portion.
• —UMLS requires acceptance of the UMLS Metathesaurus License.
• —Several instruction sources (e.g., Alpaca-derived data) are non-commercial (CC BY-NC), and dialogue datasets (ChatDoctor/HealthCareMagic) are typically research-use only.

Because of this mix, the dataset is best treated as research-use, non-commercial, with the most restrictive component license effectively governing any given use. You are responsible for reviewing and complying with the license and access terms of each original source before using or redistributing the data. The license tag is set to other to reflect this composite status.

Citation

If you use this dataset, please cite the paper:

@article{aboelenen2026medllama3,
  title   = {Med-LLaMA3: Advancing Medical Question-Answering Through Parameter-Efficient Fine-Tuning of Large Language Models},
  author  = {Abo El-Enen, Mohamed Ahmed and Ismail, Sally S. and Nazmy, Taymoor Mohamed},
  journal = {Applied Sciences},
  volume  = {16},
  number  = {12},
  pages   = {6158},
  year    = {2026},
  publisher = {MDPI},
  doi     = {10.3390/app16126158},
  url     = {https://www.mdpi.com/2076-3417/16/12/6158}
}

Please also cite the original source datasets you rely on (MedAlpaca/Medical Meadow, MedMCQA, MedQA-USMLE, PubMedQA, MIMIC-III, UMLS, HealthCareMagic/ChatDoctor, CORD-19, COVID-QA, and others listed above).

Authors & contact

Mohamed Ahmed Abo El-Enen, Sally S. Ismail, and Taymoor Mohamed Nazmy Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt.