vsamuel/qwen3-4b-clinical-note-sft-run2-checkpoint-114

Qwen3 4B Clinical Note SFT — Run 2, Checkpoint 114

This repository contains the selected LoRA adapter from run 2, checkpoint 114 of a supervised fine-tuning experiment for structured clinical-note generation. It is a submission artifact and must be loaded on top of Qwen/Qwen3-4B-Instruct-2507.

Intended use

The adapter generates a clinician-style note from a de-identified encounter prompt containing a transcript, contextual notes, patient details, and clinician-specific formatting instructions. It is intended for evaluation and research, not for clinical decision-making or unsupervised use with real patient data.

Training

• —Base model: Qwen/Qwen3-4B-Instruct-2507
• —Base revision: cdbee75f17c01a7cc42f958dc650907174af0554
• —Method: supervised fine-tuning with LoRA
• —LoRA rank / alpha / dropout: 16 / 32 / 0.05
• —Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• —Learning rate: 2e-4 with cosine scheduling and 5% warmup
• —Effective batch size: 8 (batch size 1, gradient accumulation 8)
• —Maximum sequence length: 32,768 tokens
• —Precision: bfloat16
• —Training seed: 42
• —Selected checkpoint: step 114

Checkpoint 114 was selected offline because it had the strongest validation partial-pass profile among the evaluated checkpoints across source fidelity, completeness, and instruction adherence. The held-out test set was not used for checkpoint selection.

Loading the adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

repo_id = "vsamuel/qwen3-4b-clinical-note-sft-run2-checkpoint-114"
base_id = "Qwen/Qwen3-4B-Instruct-2507"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, repo_id)

Apply the tokenizer's chat template and the same structured prompt format used during training before generation.

Limitations and safety

This is an experimental adapter trained for a narrow note-generation task. It can omit, distort, or invent clinical information and can fail clinician-specific formatting requirements. Outputs require review by a qualified human. Do not use it to diagnose, recommend treatment, or create records without appropriate validation, privacy controls, and human oversight.

Artifact integrity

SHA-256 of adapter_model.safetensors:

71a2a175a3c507fe4bae66bce78ceca39cc13647ab363b8dd15172c9b92bc567