E-motionAssistant/llama-3.2-3b-english-therapy-GGUF

LLaMA 3.2-3B English Therapy - GGUF Q4KM

Quantized GGUF version of LLaMA 3.2-3B fine-tuned for empathetic therapy conversations.

Model Details

• —Base Model: meta-llama/Llama-3.2-3B-Instruct
• —Fine-tuned on: 1,390 English therapy conversations
• —Quantization: Q4KM (4-bit)
• —File Size: ~1.8 GB (from ~6 GB original)
• —Format: GGUF (compatible with llama.cpp)

Performance Metrics

• —BLEU Score: 1.85
• —ROUGE-1: 0.29
• —ROUGE-L: 0.16
• —Perplexity: 15.97

Usage

With llama.cpp

# Download the model
wget https://huggingface.co/pavishanth-sujeevan/llama-3.2-3b-english-therapy-GGUF/resolve/main/model.gguf

# Run inference
./llama-cli -m model.gguf -p "I'm feeling anxious about my future" -n 200

With llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="model.gguf",
    n_ctx=2048,
    n_threads=4,
    n_gpu_layers=35
)

output = llm(
    "User: I'm feeling stressed about work.\nTherapist:",
    max_tokens=200,
    temperature=0.7,
    top_p=0.9
)

print(output["choices"][0]["text"])

With Streamlit

import streamlit as st
from llama_cpp import Llama

@st.cache_resource
def load_model():
    return Llama(model_path="model.gguf", n_ctx=2048, n_gpu_layers=35)

llm = load_model()
user_input = st.text_input("How are you feeling?")

if user_input:
    response = llm(f"User: {user_input}\nTherapist:", max_tokens=200)
    st.write(response["choices"][0]["text"])

Intended Use

This model is designed for:

• —Mental health chatbots
• —Therapy conversation simulation
• —Empathetic response generation
• —Research in AI-assisted counseling

Limitations

• —Not a replacement for professional therapy
• —May produce incorrect or inappropriate responses
• —Requires human oversight in production use
• —Limited to English language

Training Details

• —Training Method: LoRA fine-tuning
• —Epochs: 3
• —Learning Rate: 2e-4
• —Batch Size: 1 (gradient accumulation: 8)

License

Apache 2.0 (inherits from LLaMA 3.2 base model)