dilr/Mira-Q1

Mira-Q1 — Clinical Extraction SLM (v1)

By [DILR](https://dilr.ai) — Enterprise-grade clinical document extraction model. Fine-tuned from Qwen2.5-3B-Instruct with QLoRA to extract structured JSON from clinical documents.

Baseline Evaluation (800 gold examples)

The base model produces 0/800 valid extractions — it invents field names like labTestResults, fullName instead of following the schema. Mira-Q1 fine-tuning is essential.

Training

Schema

Extracts 10 required fields from clinical documents:

• —document_type: labreport | medicationlist | dischargesummary | pathologyreport | intakeform | progressnote | other
• —patient: {age, sex} — de-identified, no names/MRN
• —encounter: {date, department}
• —vitals[], labs[], medications[], diagnoses[], procedures[], allergies[]
• —extraction_notes

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16)
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", quantization_config=bnb, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = PeftModel.from_pretrained(base, "dilr/Mira-Q1")

messages = [
    {"role": "system", "content": "You are a clinical information extraction system..."},
    {"role": "user", "content": "Patient: 45/M\nHb 12.5 g/dL (13-17) LOW"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Base Model Zero-Shot Output (for comparison)

The base model without fine-tuning produces invalid output like:

{"patient": {"id": "87/F", "fullName": null, "dateOfBirth": "2024-01-29"},
 "labTestResults": [{"testName": "Total Cholesterol", ...}]}

Wrong field names, wrong structure, includes identifiers. Mira-Q1 fixes all of this.

Limitations

• —English only (v1)
• —Trained on synthetic data (Synthea + curated), not real clinical records
• —Every output is a draft for human review
• —Superseded by Mira-Q2 (coming soon) with 8,400 training examples and schema-as-input

License

Apache-2.0