zabir1996/alive-medical-imaging

ALIVE Medical Imaging QA Dataset

Lecture-derived question-answer corpus, retrieval index, and source materials for the ALIVE (Avatar-Lecture Interactive Video Engine) system. The dataset was built from 23 recorded lectures of an undergraduate medical imaging course and is the corpus used to fine-tune the ALIVE language model and to evaluate its retrieval and answer-generation behavior.

Layout

huggingface/
├── data/                 question-answer pairs (Alpaca-style JSONL)
│   ├── train.jsonl       4,191 examples
│   ├── val.jsonl         508 examples
│   ├── test.jsonl        508 held-out examples (open-book)
│   ├── test_full.jsonl   same 508 examples + slide metadata
│   └── stats.json        full corpus statistics
├── lectures/             slide-text .txt for the 23 medical-imaging
│                         lectures used by ALIVE (Lecture 1/ … Lecture 23/)
├── books/                course textbook ingested by the retrieval pipeline
│   └── medical_imaging_ge_wang.pdf
└── rag_store/            ready-to-load FAISS retrieval index
    ├── faiss.index
    ├── chunks.json
    └── meta.json

Splits and statistics

• —5,207 total QA pairs (train 4,191 / val 508 / test 508)
• —Stratified by lecture so every lecture appears in every split
• —Difficulty distribution: basic 1,245 / intermediate 2,013 / advanced 1,949
• —Question length: median 12 words; answer length: median 23 words

File schemas

data/train.jsonl, data/val.jsonl, data/test.jsonl — Alpaca format consumed by the LoRA fine-tuning script:

{"instruction": "<question>", "input": "LECTURE: <name>\nCONTEXT:\n<slide text>", "output": "<reference answer>"}

data/test_full.jsonl adds the same fields plus lecture_name, slide_num, difficulty, and slide_text so the evaluation scripts can stratify metrics by difficulty and lecture.

rag_store/chunks.json — one entry per chunk; slide chunks carry source = "slide", lecture/slide identifiers, and 30-second synthetic timestamps; book chunks carry source = "book", book_name, and sentinel start/end values so the temporal re-ranking term is disabled for them.

rag_store/meta.json — index configuration: embedding model, chunk counts, source-boost weight, book-chunking constants.

Loading

from datasets import load_dataset
ds = load_dataset("zabir1996/alive-medical-imaging")
print(ds["train"][0])

import faiss, json
index  = faiss.read_index("rag_store/faiss.index")
chunks = json.load(open("rag_store/chunks.json"))

License

CC BY-NC-SA 4.0. The course textbook PDF retains its original license; redistribution rights are limited to research and non-commercial use under the authors' permission. If you fork this dataset for a commercial application, contact the authors.

Code repository

System code, evaluation scripts, classroom-survey re-analysis, and the per-example output bundle used to produce every number in the ALIVE paper live in the companion GitHub repository (see the "Code and Data Availability" section of the manuscript).

Citation

@article{islam2026alive,
  title  = {ALIVE: An Avatar-Lecture Interactive Video Engine with
            Content-Aware Retrieval for Real-Time Interaction},
  author = {Islam, Md Zabirul and Manik, Md Motaleb Hossen and
            Marques, Oge and Mohamed, Hisham and Wang, Ge},
  journal= {IEEE Transactions on Learning Technologies},
  year   = {2026},
  note   = {Under review}
}

Ethical note

The classroom-survey raw responses are NOT part of this release (IRB-exempt protocol restricts redistribution of identifiable student data). Anonymized per-item summary statistics are included in the paper's supplementary material.