Maarij-Aqeel/Mental_Health_chatbot

Mental Health Empathy Bot – GPT-Neo 2.7B + LoRA

Model ID: Maarij-Aqeel/Mental_Health_chatbot A gentle, empathetic listener fine-tuned from GPT-Neo 2.7B using LoRA, trained on the Empathetic Dialogues dataset. Think of it as a kind, supportive friend—not a therapist.

Intended Use

• —For: Students, professionals, anyone needing to vent or reflect.
• —How: Web apps, bots, research prototypes.
• —Not for: Crisis support, clinical advice, or high-risk use cases.

Training Overview

• —Base Model: GPT-Neo 2.7B
• —Dataset: Empathetic Dialogues (~25k pairs)
• —Technique: LoRA (r=8, α=32), trained in 8-bit and merged to 4-bit
• —Hardware: 2× Tesla T4 on Kaggle, ~1000 steps
• —Validation Loss: ~2.02

Quick Performance

• —Perplexity: ~2.0
• —Empathy Rating: 4.2 / 5 (manual review)
• —Toxicity: <1% flagged (Perspective API)
• —Inference (CPU): ~700ms / 50 tokens
• —RAM (4-bit): ~1.3 GB

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Maarij-Aqeel/Mental_Health_chatbot", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-2.7B")

prompt = "System: You are a mental health assistant.\nUser: I'm overwhelmed.\nTherapist:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=64, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

- PEFT 0.16.0