vadimbelsky/qwen3.5-medical-ft

Qwen3.5-9B — Medical Triage LoRA Adapters

LoRA fine-tune of Qwen3.5-9B on synthetic clinical triage Q&A pairs generated from PubMed Central open-access papers. The model is specialized for emergency-medicine decision-making: triaging patients, applying clinical decision rules, and generating protocol-grounded triage recommendations.

Disclaimer — Not for clinical use. This model is a research artifact trained on synthetic data. It must not be used to inform real patient care decisions.

Model Details

• —Developed by: Vadim Belsky
• —Base model: Qwen/Qwen3.5-9B (9.4 B parameters)
• —Fine-tuning method: LoRA (PEFT 0.18.1)
• —Language: English
• —License: Apache 2.0
• —Training dataset: vadimbelsky/medical-triage-qa-50k

Quick Start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.5-9B"
adapter = "vadimbelsky/qwen3.5-medical-ft"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
model.eval()

prompt = (
    "A 67-year-old male presents with sudden onset crushing chest pain radiating to "
    "the left arm, diaphoresis, and mild dyspnea. BP 145/90, HR 102, SpO2 96%. "
    "What is the triage priority and initial management?"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, temperature=0.3, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Intended Uses

Out-of-Scope Use

• —Real-time triage of actual patients
• —Replacing trained clinicians or validated clinical decision rules
• —Domains outside emergency medicine / triage

Training Details

Dataset

~47,500 synthetic Q&A pairs generated from ~2,100 PubMed Central open-access papers covering emergency medicine, triage protocols, clinical decision rules, prehospital care, pediatric emergencies, and sepsis management.

Generation pipeline:

1. 1.Papers downloaded from PMC via fetch_papers.py
2. 2.Relevant clinical sections extracted and classified
3. 3.Q&A pairs generated by gpt-4.1-nano via NVIDIA NeMo Data Designer
4. 4.Quality-filtered by LLM judge on three axes: relevance, faithfulness, and clinical realism

Train/validation split: 95 / 5 (≈45,125 / 2,375 samples)

Hyperparameters

Training Run

Evaluation — Pre Fine-Tune Baseline

Zero-shot evaluation of the quantized base model (Q4\K\M GGUF) using lm-evaluation-harness:

Post fine-tune evaluation on these benchmarks is in progress.

Bias, Risks, and Limitations

• —Synthetic data: All training Q&A pairs are machine-generated from research papers; the model may reflect biases or errors in the source literature and generation pipeline.
• —No clinical validation: The model has not been evaluated against real patient outcomes or validated clinical guidelines.
• —English only: Trained exclusively on English-language literature.
• —Hallucination risk: Like all LLMs, this model can generate plausible-sounding but factually incorrect clinical information.
• —Narrow scope: Optimized for emergency triage scenarios; performance outside this domain is untested.

Environmental Impact

• —Hardware: NVIDIA Blackwell GB10 (DGX Spark)
• —Training time: ~15.2 hours
• —Carbon estimate: Use the ML CO₂ Impact calculator for an estimate based on your region.

Technical Specifications

Model Architecture

Qwen3.5-9B is a transformer-based causal language model with:

• —32 layers, mixed linear/full attention (Blackwell-optimized)
• —9.4 B total parameters
• —Context window: 262,144 tokens

The LoRA adapters add ~40 M trainable parameters (~0.4% of base model).

Software

Model Card Authors

Vadim Belski

Framework versions

• —PEFT 0.18.1