OncogenAI/phi-3-medium-clinical

Phi-3-Medium-Clinical

A medical domain-specific fine-tuned variant of Microsoft's Phi-3-Medium language model, optimized for clinical applications and healthcare-related tasks.

Model Details

Base Model

• —Model Name: microsoft/Phi-3-medium-4k-instruct
• —Parameter Count: 14B parameters
• —Architecture: Transformer-based language model
• —Training Data: Pre-trained on diverse web-scale data with strong reasoning and instruction-following capabilities

Fine-tuning Information

• —Fine-tuning Framework: Unsloth - High-efficiency fine-tuning toolkit
• —Medical Dataset: Clinical and healthcare-specific dataset
• —Optimization Method: LoRA (Low-Rank Adaptation) via Unsloth
• —Training Date: March 2026

Model Description

This model adapts Phi-3-medium-4k-instruct for clinical and healthcare applications by fine-tuning on curated medical datasets. The specialization enables better performance on:

• —Clinical note analysis and generation
• —Medical question answering
• —Healthcare terminology understanding
• —Patient information processing (de-identified)
• —Medical literature comprehension

Intended Use

Primary Use Cases

• —Clinical decision support and information retrieval
• —Medical text analysis and classification
• —Healthcare documentation assistance
• —Medical knowledge base queries
• —Educational purposes in healthcare settings

Limitations

• —Designed for informational and educational purposes only
• —Should not be used as a standalone clinical decision-making tool
• —Requires human medical professional validation
• —Performance may vary on specialized subspecialties
• —De-identified datasets only - respects patient privacy regulations

Training Details

Framework & Libraries

- Base: microsoft/Phi-3-medium-4k-instruct
- Fine-tuning: Unsloth
- LoRA Implementation: Efficient parameter-efficient fine-tuning
- Hardware: GPU-based training

Dataset Characteristics

• —Domain: Medical and clinical
• —Focus: Healthcare-specific terminology and concepts
• —Processing: Data preprocessing and tokenization following Phi-3 specifications
• —Format: Instruction-following format for improved generalization

Training Parameters

• —Max Sequence Length: Phi-3-medium-4k-instruct standard (typically 4096 tokens)
• —Optimization: LoRA with rank adaptation for parameter efficiency
• —Batch Size: Standard configuration for medical dataset fine-tuning
• —Evaluation: Domain-specific metrics for medical accuracy

Usage

Installation

# Install required dependencies
pip install transformers torch unsloth

# Optional: Install medical evaluation tools
pip install -r requirements.txt

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "OncogenAI/phi-3-medium-clinical"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype=torch.float16,
)

# Prepare input
prompt = "Clinical Query: Explain the pathophysiology of type 2 diabetes."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate response
outputs = model.generate(
    **inputs,
    max_length=512,
    temperature=0.7,
    top_p=0.95,
    do_sample=True,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Advanced Usage with Unsloth

from unsloth import FastLanguageModel
from transformers import TextIteratorStreamer
from threading import Thread

# Fast loading with Unsloth
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="OncogenAI/phi-3-medium-clinical",
    max_seq_length=4096,
    dtype=torch.float16,
    load_in_4bit=True,
)

# Streaming inference for long responses
streamer = TextIteratorStreamer(tokenizer, skip_special_tokens=True)
generation_kwargs = dict(
    inputs=inputs,
    streamer=streamer,
    max_length=512,
    temperature=0.7,
)

thread = Thread(target=model.generate, kwargs=generation_kwargs)
thread.start()

for text in streamer:
    print(text, end="", flush=True)

Model Architecture

• —Attention Mechanism: Multi-head self-attention
• —Positional Encoding: Rotary positional embeddings (RoPE)
• —Normalization: RMSNorm
• —Activation: SiLU (Swish)
• —Fine-tuning Method: LoRA adapters via Unsloth

Performance Characteristics

• —Inference Speed: Optimized via Unsloth for reduced latency
• —Memory Footprint: Efficient through LoRA parameter reduction
• —Medical Domain Accuracy: Enhanced through domain-specific fine-tuning
• —Token Context: Maintains Phi-3-Medium's context window

Hardware Requirements

Recommended Specifications

• —GPU Memory: 8GB+ (supports quantized inference)
• —CPU RAM: 16GB+ (for tokenization and preprocessing)
• —Storage: 15GB+ (model weights and tokenizer)

Inference Modes

• —Full Precision: 14B parameters (~28GB VRAM)
• —FP16 (Half Precision): ~14GB VRAM
• —INT8 Quantization: ~8GB VRAM
• —INT4 Quantization: ~4GB VRAM

Ethical Considerations

Responsible Use

• —This model should complement, not replace, professional medical judgment
• —Always have qualified healthcare professionals review outputs
• —Be transparent about AI usage in clinical contexts
• —Maintain patient confidentiality and data security
• —Comply with healthcare regulations (HIPAA, GDPR, etc.)

Bias and Fairness

• —Fine-tuning dataset undergoes bias assessment
• —Model outputs may reflect biases in training data
• —Users should evaluate context-specific performance
• —Continuous monitoring recommended for production use

License

[Specify your license - e.g., Apache 2.0, MIT, or custom license]

Citation

If you use this model in research, please cite:

@software{phi3_clinical_2026,
  author = {OncogenAI},
  title = {Phi-3-Medium-Clinical: A Medical Domain-Specific Fine-Tuned Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/OncogenAI/phi-3-medium-clinical}}
}

Also cite the base model and Unsloth:

@article{abdin2024phi,
  title={Phi-3 Technical Report},
  author={Abdin, Marah and others},
  journal={arXiv preprint arXiv:2404.14219},
  year={2024}
}

@software{unsloth2024,
  title={Unsloth: Efficient Fine-tuning of Language Models},
  author={Peng, Daniel Han and others},
  year={2024},
  url={https://github.com/unslothai/unsloth}
}

Limitations and Disclaimers

1. 1.Medical Accuracy: While fine-tuned on medical datasets, this model is not a substitute for professional medical advice
2. 2.Knowledge Cutoff: Performance limited to training data scope
3. 3.Specialized Domains: May have limited performance on rare conditions or cutting-edge treatments
4. 4.Privacy: Ensure input data is properly de-identified before use
5. 5.Regulatory Compliance: Users are responsible for compliance with applicable healthcare regulations

Contact & Support

For questions about this model:

• —Open an issue on the Hugging Face model page
• —Reference specific queries or behaviors for reproducibility
• —Provide context about your use case

Acknowledgments

• —Microsoft for the Phi-3-Medium base model
• —Unsloth team for the efficient fine-tuning framework
• —Medical dataset contributors and curators
• —Healthcare and AI safety researchers