Modelpublic

chenhaodev/triage-qwen3-8b-gguf

sourceHugging Faceapache-2.0updated 22d agoView on Hugging Face

0likes191downloads

Model Card

Qwen3-8B-Triage (GGUF)

This is the GGUF-quantized version of the telephone triage model, fine-tuned from Qwen/Qwen3-8B on a medical triage protocol corpus.

Quantizations

File	Quantization	Approx. size
`Qwen3-8B-Triage-f16.gguf`	f16	16.4 GB
`Qwen3-8B-Triage-q8_0.gguf`	q8_0	8.7 GB
`Qwen3-8B-Triage-q5_k_m.gguf`	q5km	5.9 GB
`Qwen3-8B-Triage-q4_k_m.gguf`	q4km	5.0 GB

Usage with llama.cpp

bash

llama-cli -m Qwen3-8B-Triage-q8_0.gguf -p "问诊..."

Usage with Ollama

Create a Modelfile pointing at one of the GGUF files:

FROM ./Qwen3-8B-Triage-q8_0.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """你是一名专业电话分诊护士，请根据患者描述的症状进行分诊，判断紧急程度并给出就医建议。"""

PARAMETER temperature 0.2
PARAMETER stop "<|im_end|>"

bash

ollama create triage -f Modelfile
ollama run triage

Or, since this repo is a GGUF hub repo, you can pull a specific quant directly:

bash

ollama pull hf.co/chenhaodev/triage-qwen3-8b-gguf:q8_0