ShivomH/Elixir-MentalHealth-LoRA

Model Details

This is just a LoRA Adapter, please navigate to ShivomH/Elixir-MentalHealth-3B to access the merged model with a guided inference script.

Model Description

Elixir-MentalHealth is a fine-tuned version of Meta-Llama-3.2-3B-Instruct, adapted using QLoRA on a curated dataset of single-turn and multi-turn mental health support conversations. The model is designed to provide empathetic, safe, and supportive responses while maintaining clear professional boundaries.

⚠️ Disclaimer: This model is not a replacement for professional mental health services. Always seek help from licensed professionals in crisis situations.

Primary Use Cases:

• —Mental health support chats
• —Stress and Anxiety management conversations
• —Empathetic listening, encouragement and general guidance
• —Psychoeducational tips (e.g., mindfulness, coping strategies, depression support)

Out-of-Scope Use (should NOT be used for):

• —Medical diagnosis or treatment planning
• —Emergency mental health intervention (e.g., suicide prevention crisis line replacement)
• —Legal, financial, or unrelated domains

This model is best suited for research, prototyping, and supportive chatbot applications where professional disclaimers and human oversight are always present.

How to Get Started with the Model

# Load model with LoRA
from peft import PeftModel, PeftConfig

lora_model = "ShivomH/Elixir-MentalHealth-3B"
base_model = "meta-llama/Llama-3.2-3B-Instruct"

# Load configuration
peft_config = PeftConfig.from_pretrained(lora_model)

# Load base model
inference_model = AutoModelForCausalLM.from_pretrained(
    peft_config.base_model,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load LoRA weights
inference_model = PeftModel.from_pretrained(inference_model, lora_model)

# Load tokenizer
inference_tokenizer = AutoTokenizer.from_pretrained(lora_model)

📊 Dataset Details

• —Dataset Source: ShivomH/MentalHealth-Support
• —Size: 25,000 conversations
• —Training Split: 23,750 (95%)
• —Validation Split: 1,250 (5%)
• —Multi-Turn Conversations: 16,000
• —Long Single-Turn Conversations: 8,000
• —Short Single-Turn Conversations: 1,000
• —Total tokens: ~17M
• —Mean: ~700 tokens
• —Data format: (.jsonl) Messages List with Roles and Content

General Details

• —Developed by: Shivom Hatalkar
• —Funded by: Shivom Hatalkar
• —Model type: NLP Text Generation LLM
• —Language(s) (NLP): English
• —License: llama3.2
• —Base Model: meta-llama/Llama-3.2-3B-Instruct

Model Sources

<!-- Provide the basic links for the model. -->

• —Repository: [More Information Needed]
• —Paper [optional]: [More Information Needed]
• —Demo [optional]: [More Information Needed]

Training Details

Please visit the Merged model ShivomH/Elixir-MentalHealth-3B page for detailed Training details.

Results

Please visit the Merged model ShivomH/Elixir-MentalHealth-3B page for viewing the testing samples.

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Framework versions

• —PEFT 0.17.1