logos-flux/Qwen3-4B-Instruct-2507-heretic-GGUF

Qwen3-4B-Instruct-2507-heretic-GGUF

GGUF quantized versions of p-e-w/Qwen3-4B-Instruct-2507-heretic from "The Bestiary" collection.

Model Description

This is a GGUF conversion of the Qwen3-4B-Instruct-2507-heretic model, which is an abliterated (uncensored) version of Alibaba's Qwen2.5 4B Instruct model. The model has had its refusal mechanisms removed, making it more willing to engage with any prompt.

Original Model: p-e-w/Qwen3-4B-Instruct-2507-heretic Collection: The Bestiary by p-e-w

Quantization Formats

This repository contains 4 quantization levels:

Recommended: Q4_K_M for most users (best balance of quality and size)

Usage

With Ollama

1. 1.Download the GGUF file you want to use
2. 2.Create a Modelfile:

FROM ./qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system<|im_sep|>{{ .System }}<|im_end|>{{ end }}{{ if .Prompt }}<|im_start|>user<|im_sep|>{{ .Prompt }}<|im_end|>{{ end }}<|im_start|>assistant<|im_sep|>{{ .Response }}<|im_end|>"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192

1. 1.Import to Ollama:

ollama create qwen3-4b-heretic:Q4_K_M -f Modelfile

1. 1.Run:

ollama run qwen3-4b-heretic:Q4_K_M

With llama.cpp

./llama-cli -m qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf -p "Your prompt here" -n 512

With Open WebUI

Once imported to Ollama, the model will automatically appear in the Open WebUI model dropdown.

Conversion Details

• —Converted using: llama.cpp (latest)
• —Conversion date: 2025-11-21
• —Base format: FP16 GGUF
• —Quantization method: llama-quantize

Important Note

This is an uncensored model with refusal mechanisms removed. Use responsibly and in accordance with applicable laws and regulations.

License

Inherits the Apache 2.0 license from the base Qwen model.

Credits

• —Original model: Alibaba (Qwen2.5)
• —Abliteration: p-e-w (The Bestiary)
• —GGUF conversion: cybrown