zainkhanz/qwen3-4b-toxic-Q4_K_M-GGUF

Qwen3-4B Toxic — Q4KM GGUF

GGUF quantized version of [zainkhanz/qwen3-4b-toxic-unsloth](https://huggingface.co/zainkhanz/qwen3-4b-toxic-unsloth).

Fine-tuned from [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) with Unsloth (SFT → DPO) to be more direct and low-refusal.

This repo contains the Q4_K_M GGUF for use with llama.cpp, Ollama, LM Studio, KoboldCpp, etc.

File

Related checkpoints

Behavior

Compared with the stock Qwen3-4B-Instruct model, this fine-tune is trained to:

• —Answer the user’s request more directly
• —Avoid generic policy-style refusals on the training distribution
• —Stay conversational and straightforward

It still inherits normal LLM limits (hallucination, bias, factual errors). You are responsible for how you use the outputs.

Quick start

llama.cpp

./llama-cli -m qwen3-4b-toxic-Q4KM.gguf -p "Your question here" -n 512 --temp 0.7 -c 8192

Ollama

ollama run hf.co/zainkhanz/qwen3-4b-toxic-Q4KM-GGUF

Or create a Modelfile:

FROM ./qwen3-4b-toxic-Q4KM.gguf

TEMPLATE """{{ if .System }}<|imstart|>system {{ .System }}<|imend|> {{ end }}{{ if .Prompt }}<|imstart|>user {{ .Prompt }}<|imend|> {{ end }}<|imstart|>assistant {{ .Response }}<|imend|>"""

PARAMETER temperature 0.7 PARAMETER stop "<|imstart|>" PARAMETER stop "<|imend|>"

Then:

ollama create qwen3-toxic -f Modelfile ollama run qwen3-toxic

LM Studio / KoboldCpp

Load qwen3-4b-toxic-Q4KM.gguf and use the Qwen / ChatML template.

Chat template (Qwen)

<|imstart|>system {system}<|imend|> <|imstart|>user {user}<|imend|> <|im_start|>assistant

Training summary

License

Apache 2.0 (same as base Qwen3).