CoolFace
Modelpublic

Jaamie/gemma_mental_health_qlora_v2

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes11downloads
Model Card

🧠 Gemma Mental Health QLoRA v2

A fine-tuned version of google/gemma-2-9b-it for mental health diagnosis using instruction-style QLoRA tuning. This model takes in user statements and predicts the most likely mental disorder in a structured dialogue format.

🔧 Model Details

  • —Base Model: `google/gemma-2-9b-it`
  • —Fine-Tuning Method: QLoRA (4-bit quantization with bitsandbytes)
  • —Tokenizer: ✅ Included
  • —LoRA Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj"]
  • —Sequence Format:

Output format

  • —User: <statement> Diagnosed Mental Disorder: <PredictedMentalHealth>

🧪 Use Cases

  • —🧠 Mental health Q&A assistant
  • —🗨️ Conversational diagnosis suggestion
  • —📚 NLP research and experimentation
⚠️ Disclaimer: This model is for research and educational purposes only. It is not intended for use in real-world clinical diagnosis without medical supervision.

💻 How to Use

python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer and base + adapter model
tokenizer = AutoTokenizer.from_pretrained("Jaamie/gemma_mental_health_qlora_v2")
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "Jaamie/gemma_mental_health_qlora_v2")

# Inference example
prompt = "User: I can't sleep and my thoughts are spiraling out of control.\nDiagnosed Mental Disorder:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
  outputs = model.generate(**inputs, max_new_tokens=30)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))


🏋️ Training Details
Epochs: 2

Batch Size: 4 (with gradient_accumulation_steps = 2)

Max Length: 512

Quantization: 4-bit QLoRA (NF4) with bitsandbytes

Precision: bf16


# Evaluation Results

Metric	         Score
Training Loss	 3.74
Validation Loss	 3.79
Total Examples	 ~22,000


The LLM has been trained on a sample of data from the dataset containing balanced instruction-style dataset with labeled disorders.

Mental Health Class	Sample Count
Depression	4,000
Anxiety	4,000
Suicidal Thoughts	3,000
Personality Disorder	2,000
Bipolar	2,000
Stress	2,000
Normal	5,000

# Contact
Created by Jaamie Maarsh Joy Martin

🌐 https://www.linkedin.com/in/jaamie-maarsh-joy-martin/

📧 jaamiemaarsh@gmail.com