W4A16
Datasets
All datasets matching “W4A16”daily-paper-2026-07-30-moe-w4a16-pruned-30b-single-gpu
Memory-Bounded W4A16 Quantization of Pruned 30B Mixture-of-Experts LLMs on a Single Host-Constrained GPU
TL;DR — Memory-bounded pipeline compresses Qwen3-Coder-30B-A3B (61 GB) to 12.7-16.9 GB (3.6x-4.8x) on a single H200 GPU under a 32 GiB host-RAM ceiling by streaming expert pruning and per-layer RTN W4A16, with WikiText-2 perplexity degradation of +3.9% to +30.5% — no full model ever resident in CPU memory.
ThakiCloud AI Research · 2026-07-30 · 📝 Tech blog (KO)… See the full description on the dataset page: https://huggingface.co/datasets/thaki-AI/daily-paper-2026-07-30-moe-w4a16-pruned-30b-single-gpu.dsv4-flash-w4a16-rtxpro6000-image
canada-quant/dsv4-flash-w4a16-rtxpro6000-image
Pre-built Docker image (canada-quant/dsv4-w4a16-rtxpro6000:v1) that serves
canada-quant/DeepSeek-V4-Flash-W4A16-FP8-MTP
on RTX PRO 6000 Blackwell (SM 12.0a) out of the box.
Why this image exists: the W4A16 artifact needs a tightly-pinned vLLM build
(jasl/vllm@27fd665b + canada-quant BF16-MTP cherry-pick + ~13 layers of
dependency/patch fixes) to serve correctly on consumer/server Blackwell.
Rebuilding all that on a fresh box is ~25… See the full description on the dataset page: https://huggingface.co/datasets/canada-quant/dsv4-flash-w4a16-rtxpro6000-image.hy3-w4a16-mtp-calibration
Hy3 W4A16-MTP — Calibration Set
The exact 512-sample calibration blend used to GPTQ-quantize
canada-quant/hy3-w4a16-mtp
(a W4A16 quantization of tencent/Hy3).
Published for full reproducibility of the quantization pipeline.
Why a blend (not chat-only)
INT4 weight quantization degrades most on code and tool-call-shaped tokens. A chat-only
calibration set (e.g. pure ultrachat) under-samples exactly the routed experts those tokens
activate. This set deliberately… See the full description on the dataset page: https://huggingface.co/datasets/canada-quant/hy3-w4a16-mtp-calibration.dsv4-flash-w4a16-spark-image
vllm-w4a16-dsv4:exp — pre-built Docker image for DGX Spark TP=2
This dataset hosts a pre-built OCI tarball of the Docker image used to serve
pastapaul/DeepSeek-V4-Flash-W4A16-FP8
on dual DGX Spark GB10 (SM 12.1a) — for users who can't build the image
themselves due to network constraints.
The image is also fine for 2× RTX PRO 6000 Blackwell Server (SM 12.0).
What's inside
vLLM build pinned to:
jasl/vllm@ds4-sm120-experimental (HEAD c05638d70 after cherry-pick) —… See the full description on the dataset page: https://huggingface.co/datasets/canada-quant/dsv4-flash-w4a16-spark-image.awq-kivi-w4a16kv8
AWQ W4A16 + KIVI-INT8 KV (Llama-3.1-8B-Instruct)
w4_awq_llama_31_8b/ — W4 weights
The 4-bit weights of the AWQ W4A16 model (asymmetric / standard AWQ, group size 128).
Stored per layer: layer_0.safetensors … layer_31.safetensors + embeddings.safetensors.
The 7 linears per layer are quantized (packed int4 codes + fp16 per-group scale + uint8 per-group zero); everything else stays fp16:
key
dtype
shape
*.qweight (2 int4 codes / byte; lo nibble = even… See the full description on the dataset page: https://huggingface.co/datasets/jsyeom/awq-kivi-w4a16kv8.