Modelpublic

TheHouseOfTheDude/Qwen3.6-27B-INT8

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

10likes390downloads

Model Card

Qwen3.6-27B-INT8 — PTQ Quantized (W8A16)

Overview

Post-Training Quantized (PTQ) version of Qwen/Qwen3.6-27B.

—No calibration dataset
—One-shot quantization
—Uses llmcompressor

Quantization

—Scheme: W8A16
—Weights: INT8
—Activations: FP16/BF16
—Targets: Linear layers only

Ignored:

—lm_head
—visual
—linear_attn
—mtp

KLD Results:

—Mean KLD: 0.009441

Usage

bash

vllm serve TheHouseOfTheDude/Qwen3.6-27B-INT8 \
  --quantization compressed-tensors

Notes

—Requires vLLM
—Not for vanilla Transformers