AserLompo/khp-youth-mental-health-guardrail

KHP Youth Mental Health Safety Guardrail Dataset

A synthetic multi-turn conversational dataset for training and evaluating input guardrails for AI assistants serving youth in mental distress. The dataset targets 9 distress signal categories and provides a binary high-risk label for classification, decoupled from the individual signal flags.

GitHub repository: Aser97/Guardrail-For-Agents

Dataset Summary

The dataset is perfectly balanced (50/50 high-risk / low-risk) and covers English, French, and mixed EN/FR conversations.

Motivation

There are no existing datasets that combine:

• —Multi-turn conversational format (not single posts/messages)
• —Youth-specific register (age 13–17, informal language, slang)
• —Bilingual EN/FR coverage
• —Nine distinct distress signal categories with individual labels
• —A high-risk label explicitly decoupled from simple signal OR-logic

This dataset was built from scratch using a combination of adversarial and augmentation techniques to fill that gap.

Signal Taxonomy

Each row carries 9 binary signal flags (s_* columns) alongside a global label field:

Important: label = 1 (high-risk) is not equivalent to any(s_* == 1). Signals can be present without warranting escalation (e.g. hypothetical framing, third-party concern without acute risk). The label field reflects a holistic assessment of whether the conversation requires urgent support.

Data Generation

The dataset was built using four complementary techniques:

All generated rows were verified by a Claude Sonnet jury scoring realism, subtlety, and signal presence.

Schema

{
    "text":               str,   # Full multi-turn conversation (user/assistant turns)
    "label":              int,   # 0 = low-risk, 1 = high-risk
    "source":             str,   # Generation technique (see table above)
    "primary_signal":     str,   # Main distress signal targeted during generation
    "escalation_stage":   str,   # Stage of signal escalation (where applicable)
    "register":           str,   # Language register / slang profile used
    "language":           str,   # "en", "fr", or "mix"
    "persona_id":         str,   # Persona profile identifier
    "category":           str,   # Thematic category of the stressor
    # Individual signal flags (0 or 1):
    "s_burden_language":     int,
    "s_finality_language":   int,
    "s_escape_framing":      int,
    "s_hopelessness":        int,
    "s_active_self_harm":    int,
    "s_immediate_safety":    int,
    "s_self_image_crisis":   int,
    "s_third_party_concern": int,
    "s_testing":             int,
}

Usage

from datasets import load_dataset

ds = load_dataset("Aser97/khp-youth-mental-health-guardrail")

# Binary classification (high-risk / low-risk)
train = ds["train"]
print(train[0]["text"])
print("High-risk:", train[0]["label"])

# Multi-label signal detection
print("Hopelessness:", train[0]["s_hopelessness"])
print("Immediate safety:", train[0]["s_immediate_safety"])

Recommended Model

The dataset was used to fine-tune Qwen2.5-7B-Instruct with LoRA (r=16, α=32) on a 9-head multi-label classification layer, followed by a logistic regression aggregation head over the 9 signal probabilities to produce the final high-risk binary prediction.

See the GitHub repository for training scripts and the full pipeline.

Evaluation

Evaluated on an external real-world youth mental health corpus (94 samples):

Confusion matrix: TP=37, FP=8, TN=44, FN=5. All 5 false negatives are qualitatively borderline cases where annotator agreement is low.

Ethical Considerations

• —All conversations are fully synthetic — no real user data was used.
• —The dataset is intended for safety classifier training only, not for generating harmful content.
• —Signal categories and conversation content involve sensitive topics (self-harm, suicidal ideation, abuse). Researchers using this dataset should follow appropriate ethical review processes.
• —The high-risk label reflects a calibrated safety threshold, not a clinical diagnosis.

Citation

If you use this dataset, please cite:

@dataset{lompo2026khp,
  author    = {Lompo, Boammani Aser},
  title     = {{KHP Youth Mental Health Safety Guardrail Dataset}},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/datasets/AserLompo/khp-youth-mental-health-guardrail}
}

The following works were instrumental in the data generation pipeline:

• —Li et al. (2023). CAMEL: Communicative Agents for "Mind" Exploration. NeurIPS. arXiv:2303.17760
• —Chao et al. (2023). Jailbreaking Black Box LLMs in Twenty Queries. arXiv:2310.08419
• —Liu et al. (2021). Towards Emotional Support Dialog Systems. ACL 2021. arXiv:2106.01144

License

MIT — free to use for research and non-commercial applications. Please review ethical considerations above before use.