anshpunia8597/mistral-7b-medical-qa-finetuned

Mistral-7B Medical QA — LoRA Fine-Tuned

A medical-domain adaptation of Mistral-7B-Instruct-v0.2, fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) with LoRA for medical question answering and healthcare AI research.

Important: This model is intended for research, experimentation, and educational use only. It has not been clinically validated and must not be used as a substitute for professional medical advice, diagnosis, treatment, or clinical decision-making.

Model Details

This model is a domain-adapted version of mistralai/Mistral-7B-Instruct-v0.2.

Rather than training a foundation model from scratch, this project applies Low-Rank Adaptation (LoRA) using the Hugging Face PEFT framework to specialize Mistral-7B-Instruct-v0.2 for medical question-answering tasks.

Basic Information

• —Developer: Ansh Punia
• —Project type: Independent AI/ML research and engineering project
• —Model type: PEFT/LoRA adapter for a decoder-only causal language model
• —Base model: mistralai/Mistral-7B-Instruct-v0.2
• —Base model size: Approximately 7 billion parameters
• —Fine-tuning method: Low-Rank Adaptation (LoRA)
• —Framework: Hugging Face Transformers + PEFT
• —Primary task: Medical-domain question answering and text generation
• —Language: Primarily English
• —Clinical validation: None
• —Status: Experimental / research

Intended Uses

This model is intended primarily for:

• —Medical NLP research
• —Medical question-answering experiments
• —LLM domain-adaptation research
• —PEFT and LoRA experimentation
• —Healthcare conversational-AI prototypes
• —Educational demonstrations
• —RAG and agentic-AI research

Generated responses should be treated as machine-generated research outputs rather than medical advice.

Out-of-Scope Use

This model should not be independently used for:

• —Clinical diagnosis
• —Treatment recommendations
• —Medication prescribing or dosing
• —Emergency medical decision-making
• —Autonomous clinical decision support
• —Replacement of qualified healthcare professionals
• —High-stakes patient-specific medical decisions

The model has not been clinically validated and may generate inaccurate, incomplete, outdated, misleading, or hallucinated information.

Bias, Risks, and Limitations

Potential limitations include:

• —Hallucinated medical information
• —Incorrect or incomplete responses
• —Overconfident answers
• —Bias inherited from the base model or training data
• —Uneven performance across medical specialties
• —Potentially outdated medical knowledge
• —Sensitivity to prompt wording
• —Lack of patient-specific clinical context
• —Lack of prospective clinical validation

Researchers should independently evaluate the model before downstream use and verify important outputs against authoritative medical sources.

How to Get Started

This repository contains a PEFT/LoRA adapter. Load the original Mistral model and then apply the adapter.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_id = "anshpunia8597/mistral-7b-medical-qa-finetuned"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    adapter_id
)

messages = [
    {
        "role": "user",
        "content": "Explain the major risk factors associated with hypertension."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        inputs,
        max_new_tokens=300,
        temperature=0.7,
        do_sample=True
    )

response = tokenizer.decode(
    outputs[0][inputs.shape[-1]:],
    skip_special_tokens=True
)

print(response)

Training Details

Training Objective

The objective of this project was to explore whether a general-purpose instruction-following language model could be efficiently adapted toward medical question answering using parameter-efficient fine-tuning.

LoRA was used instead of full-parameter fine-tuning to reduce the number of trainable parameters and lower memory and compute requirements.

Training Data

The model was fine-tuned using medical-domain question-answering data prepared for supervised fine-tuning.

Detailed dataset documentation will be added after verification of the original training data, provenance, licensing, preprocessing pipeline, and train/validation/test splits.

Verified Training Information

• —Method: LoRA
• —Framework: Hugging Face PEFT
• —Base model: mistralai/Mistral-7B-Instruct-v0.2
• —Model family: Mistral
• —Training objective: Medical-domain question answering
• —PEFT version: 0.18.0

Exact LoRA rank, alpha, dropout, target modules, learning rate, batch size, epochs, optimizer, scheduler, and training precision should be added after verifying the original training configuration.

Evaluation

A comprehensive quantitative benchmark comparing this adapter against the original mistralai/Mistral-7B-Instruct-v0.2 model has not yet been published.

No claim of clinically validated performance improvement is made.

Future evaluation may include:

• —Factual correctness
• —Medical relevance
• —Response completeness
• —Hallucination rate
• —Instruction adherence
• —Medical QA benchmark performance
• —Human or expert evaluation

Technical Specifications

The underlying model is Mistral-7B-Instruct-v0.2, an instruction-tuned decoder-only transformer language model.

This repository contains PEFT/LoRA adapter weights for medical-domain question answering. The project does not claim to have trained a 7-billion-parameter foundation model from scratch.

Open-Source Contribution

The adapter has been publicly released through Hugging Face to make the work accessible to researchers, students, and developers.

Where permitted by applicable licenses, users can inspect, download, evaluate, and build upon the adapter.

Future Work

• —Rigorous evaluation against the original Mistral model
• —Evaluation on established medical QA benchmarks
• —Quantitative hallucination analysis
• —Improved dataset documentation
• —Expanded medical-domain coverage
• —Retrieval-Augmented Generation experiments
• —Improved safety and uncertainty handling
• —Agentic medical-AI research

Disclaimer

This model is provided for research and educational purposes only.

It is not a medical device and has not been validated or approved for clinical use.

The model may produce incorrect, incomplete, misleading, outdated, or fabricated information. Generated content should not be relied upon for diagnosis, treatment, medication decisions, emergency decisions, or other medical care.

Always consult qualified healthcare professionals and authoritative medical sources for medical decisions.

Model Card Author

Ansh Punia

Independent AI/ML developer working on large language models, medical AI, retrieval-augmented generation, and applied artificial intelligence.

Framework Version

• —PEFT 0.18.0