Hriday75/qwen2.5-3b-oncology-dpo-aligned

Qwen2.5-3B Oncology Expert (DPO Aligned)

Model Details

Model Description

This is the final, fully aligned (Phase 3) LoRA adapter for the Oncology expert. While Phase 1 taught the model cancer biology and Phase 2 taught it to chat with patients, Phase 3 utilizes Direct Preference Optimization (DPO) to align the model's outputs for structural safety, factual adherence, and proper clinical formatting.

• —Developed by: Hriday75
• —Model type: LoRA Adapter (PEFT)
• —Language: English
• —Finetuned from base model: unsloth/Qwen2.5-3B-bnb-4bit (via Phase 2 Oncology Chat LoRA)
• —Training Stage: Phase 3 (Direct Preference Optimization - DPO)

Uses

This model is intended to be used in high-fidelity clinical routing pipelines where structural output constraints are just as important as clinical accuracy. It is designed to heavily penalize hallucinated reasoning paths or structurally invalid medical data formatting when parsing oncology reports, pathology results, or chemotherapy regimens.

Training Details

Training Data

This adapter was aligned using the ai-galileo/clinical-notes-to-fhir dataset. This dataset acts as a preference learning corpus, providing the model with chosen (structurally valid medical outputs) and rejected (annotated structural or formatting errors) examples to learn from.

Training Procedure

Trained using unsloth and the TRL DPOTrainer. DPO requires an extremely low learning rate to subtly shift the model's generation probabilities toward the "chosen" format without destroying the dense scientific knowledge learned in Phases 1 and 2.

How to Get Started with the Model

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# 1. Load the base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-3B-bnb-4bit")

# 2. Attach this DPO Aligned Oncology Adapter
model = PeftModel.from_pretrained(base_model, "Hriday75/qwen2.5-3b-oncology-dpo-aligned")

# 3. Format your chat prompt
messages = [
    {"role": "system", "content": "You are a helpful, empathetic oncology expert."},
    {"role": "user", "content": "Please review this surgical pathology report and format the tumor staging into a structured clinical output."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)