sara4dev/unsloth-nemotron-3-nano-medical-qa-lora

Nemotron-Nano MedMCQA LoRA Adapter

A LoRA (Low-Rank Adaptation) adapter fine-tuned on the MedMCQA dataset for medical question answering.

Notice: Licensed by NVIDIA Corporation under the NVIDIA Nemotron Open Model License.

Model Details

Model Description

This is a LoRA adapter trained on top of NVIDIA's Nemotron-3-Nano-30B model using the MedMCQA dataset. The adapter specializes the base model for answering medical multiple-choice questions, covering topics from AIIMS and NEET PG medical entrance exams.

• —Base Model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
• —Model Type: LoRA Adapter (PEFT)
• —Language: English
• —License: NVIDIA Nemotron Open Model License
• —Fine-tuned from: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16

Model Sources

• —Base Model: NVIDIA Nemotron-3-Nano-30B
• —Training Dataset: openlifescienceai/medmcqa

Uses

Direct Use

This adapter is designed for medical question answering tasks, particularly multiple-choice questions in the medical domain. It can be used for:

• —Answering medical knowledge questions
• —Educational medical Q&A systems
• —Medical exam preparation assistance

Out-of-Scope Use

This model should NOT be used for:

• —Clinical diagnosis or medical decision-making
• —Replacing professional medical advice
• —Patient care or treatment recommendations
• —Any use case where errors could cause harm to individuals

Bias, Risks, and Limitations

• —Not a Medical Device: This model is not FDA-approved or validated for clinical use
• —Training Data Bias: The model is trained on Indian medical entrance exam data (AIIMS/NEET PG), which may not generalize to all medical contexts
• —No Guarantee of Accuracy: The model can and will make mistakes on medical questions
• —English Only: The model is trained on English medical content only

Recommendations

• —Always verify medical information with qualified healthcare professionals
• —Use this model for educational or research purposes only
• —Do not rely on this model for any medical decisions

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
base_model_id = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16"
adapter_id = "YOUR_USERNAME/nemotron-nano-medmcqa-lora"  # Update with your repo

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype="auto",
    device_map="auto",
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_id)

# Example usage
prompt = """<|im_start|>system
You are a medical expert. Answer the multiple choice question.<|im_end|}
<|im_start|>user
Question: Which drug causes cinchonism?

A) Aspirin
B) Quinine
C) Paracetamol
D) Ibuprofen<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

The model was fine-tuned on the MedMCQA dataset, which contains:

• —194k+ multiple-choice questions from Indian medical entrance exams
• —Questions from AIIMS and NEET PG exams
• —Coverage of 2,400+ healthcare topics across 21 medical subjects
• —Includes explanations for correct answers

Training Procedure

Training Hyperparameters

LoRA Configuration

{
  "r": 16,
  "lora_alpha": 32,
  "lora_dropout": 0.05,
  "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj", "up_proj", "down_proj"],
  "task_type": "CAUSAL_LM",
  "bias": "none"
}

Technical Specifications

Compute Infrastructure

• —Training Framework: Unsloth + TRL (SFTTrainer)
• —PEFT Version: 0.18.1

License

This model is released under the NVIDIA Nemotron Open Model License.

Key terms:

• —Commercial use: Permitted
• —Derivative works: Permitted
• —Distribution: Permitted with attribution

Citation

If you use this adapter, please cite the base model and dataset:

@misc{nemotron3nano,
  title={NVIDIA Nemotron-3-Nano-30B},
  author={NVIDIA},
  year={2025},
  url={https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16}
}

@article{pal2022medmcqa,
  title={MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering},
  author={Pal, Ankit and Umapathi, Logesh Kumar and Sankarasubbu, Malaikannan},
  journal={arXiv preprint arXiv:2203.14371},
  year={2022}
}

Framework Versions

• —PEFT: 0.18.1
• —Transformers: Compatible with transformers 4.x
• —Unsloth: Used for training optimization

Framework versions

• —PEFT 0.18.1