Modelpublic

DESUCLUB/Qwen3-14B-v0.2-deepresearch-no-think-100-step-bf16-quantized.w8a8

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes22downloads

Model Card

DESUCLUB/Qwen3-14B-v0.2-deepresearch-no-think-100-step-bf16-quantized.w8a8

This is a custom W8A8 quantized version of jan-hq/Qwen3-14B-v0.2-deepresearch-no-think-100-step.

Quantization Details

—Method: Custom W8A8 (8-bit weights, 8-bit activations on vLLM)
—Weight precision: INT8
—Scale precision: BF16
—Quantization: Symmetric per-channel
—Zero points: None (symmetric)

Model Structure

The quantized model contains:

—.weight: INT8 quantized weights
—.weight_scale: BF16 scale parameters (trainable)
—Standard embedding and normalization layers in original precision

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

# Note: This requires custom quantization code to load properly
model = AutoModelForCausalLM.from_pretrained("DESUCLUB/Qwen3-14B-v0.2-deepresearch-no-think-100-step-bf16-quantized.w8a8")
tokenizer = AutoTokenizer.from_pretrained("DESUCLUB/Qwen3-14B-v0.2-deepresearch-no-think-100-step-bf16-quantized.w8a8")