baa-ai/Kimi-K2.6-RAM-447GB-GGUF

Kimi-K2.6 RAM GGUF — 447 GB (Q3_K experts)

Mixed-precision GGUF quantization of moonshotai/Kimi-K2.6, produced with the RAM (Resource-Aware Mixed-precision) pipeline.

Higher-quality variant. For a lower-footprint build, see baa-ai/Kimi-K2.6-RAM-344GB-GGUF.

File

Method

Quantization bit depths are assigned per-tensor using sensitivity probing rather than a uniform scheme. Each attention tensor receives bits proportional to how much its output diverges under quantization noise, measured across 8 random probes. Expert tensors (384 routed experts × 60 MoE layers) are quantized uniformly at Q3_K.

Architecture: DeepSeek-V3 decoder (kimik2), 61 layers, 384 routed experts / 8 active per token, MLA attention, hiddensize=7168, 1.03T total parameters.

Usage (llama.cpp)

llama-cli \
  -m kimi-k2.6-ram-447gb.gguf \
  -c 8192 \
  --temp 0.6 \
  -p "You are a helpful assistant."

Notes

• —No importance matrix (imatrix) was used — the 1.1 TB Q8_0 intermediate required for imatrix generation exceeds available RAM for inference on this machine. Bit allocation from sensitivity probing provides the primary quality signal.
• —The original model uses Neural Magic compressed-tensors INT4 for expert weights. These are dequantized and re-quantized to GGUF format during conversion.