CoolFace
Modelpublic

Academica/Academica-Qwen3-8B-3Bit-Heretic

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
0likes18downloads
Model Card

Academica-Qwen3-8B-3Bit-Heretic

Qwen3-8B with Heretic refusal-direction ablation and Q2_K quantization.

Academica produced the ablated model and all three published quantizations. This 3-bit release balances memory use, throughput, and generation quality.

Model details

PropertyValue
Base modelQwen/Qwen3-8B
ModificationHeretic ablation after 20 Optuna trials
QuantizationQ2_K
File size3.1 GB
Measured throughput58 tokens/second
Measured perplexity1.86
Refusal evaluation18 refusals from 100 prompts
Evaluation hardwareNVIDIA DGX Spark, GB10 Blackwell, 128 GB unified memory

The measurements describe the published artifact on the stated hardware. Results can differ with prompts, runtimes, and sampling settings.

Files

  • —qwen3-8b-heretic-q2_k.gguf: Q2_K model weights.

Use

python
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="Academica/Academica-Qwen3-8B-3Bit-Heretic",
    filename="qwen3-8b-heretic-q2_k.gguf",
)
bash
llama-server -m qwen3-8b-heretic-q2_k.gguf --host 0.0.0.0 --port 8080 -ngl 99

Limitations

Q2_K compression can reduce accuracy relative to larger quantizations. Validate the model against the intended task and runtime.

The ablation reduces refusal behavior. It does not establish factual accuracy, safety, or suitability for regulated decisions.

Related releases