ningpy/intent-detection-V2.3

Intent Detection V2.3 (V12.9)

Health-app user intent classifier + safety router fine-tuned from Qwen2.5-7B-Instruct. Routes user messages into 5 states: PASS, REDFLAG, BLOCK, STEER, or COEXIST (emergency + personal-dose).

Performance (val set: 2518 samples)

Improvements vs V11 (single-label):

• —Safety F1: 90.9% → 92.3% (+1.4pp)
• —REDFLAG F1: 90.0% → 94.1% (+4.1pp)
• —New multi-label capability: 76.2% coexist recall (v11 was 0%)

Output Schema (V12.9 flat)

6 fixed keys, deterministic order:

{
  "intent": "HA|HCN|MT|PI|OTHER",
  "redflag": true|false,
  "redflag_code": "RF-1_GENERAL|RF-2_PEDIATRIC|RF-3_OBSTETRIC|RF-4_ELDERLY|RF-5_IMMUNOCOMP|\"\"",
  "block": "BLOCK_HARD|STEER_SOFT|PASS|NONE",
  "block_code": "OOS_H_*|OOS_S_*|PASS_*|\"\"",
  "entities": {
    "drugs": ["<verbatim from user text>"],
    "diseases": ["..."],
    "symptoms": ["..."]
  }
}

Entities are verbatim substrings of the user message (no translation/normalization).

5 Valid State Combinations

Training data

• —54,404 samples (English + Chinese + Bruneian Malay)
• —Base 27K + STEER×3 + coexist×8 + patch13×3 (targeted narrow rules) + patch16×5 (anti-drift + narrow)
• —57.1% samples have non-empty entities (GPT-4o back-filled)

Usage with vLLM

python -m vllm.entrypoints.openai.api_server \
    --model ningpy/intent-detection-V2.3 \
    --served-model-name intent \
    --dtype float16 \
    --port 8001

Client sample

See https://git.evyd.tech/peiyan.ning/intent-model-v12/tree/main/sample for a Python client with routing + response templates.

Recommended request settings:

• —temperature: 0.0
• —max_tokens: 200
• —stop: [] (rely on model's natural <|im_end|>)

Downstream routing