CoolFace
Modelpublic

StargazerLabs/Qwen3.8-32B-Jumbo-3bit

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face
1likes218downloads
Model Card

Qwen3.8-32B-Jumbo (3-bit)

3-bit quantization of Qwen3.8-32B-Jumbo.

A 76-layer, 31.9B-parameter model created by transplanting the three most-drifted organs from Qwen3.6-27B into Qwen3.8-27B. See the bf16 model card for the full organ selection methodology.

At 3-bit, the 76-layer Jumbo fits in ~16 GB with speculative decoding — roughly the same memory envelope as a 7B model at bf16, but with 76 layers and 32B-class capability.

Key Stats

  • —Parameters: ~31.9B
  • —Layers: 76
  • —Precision: 3-bit (mlx-lm quantized, group_size 64)
  • —Peak memory: ~16.4 GB (with MTP drafter loaded)
  • —MTP Compatible: ~82% drafter acceptance, ~1.6x speculative decoding speedup

Usage (MLX)

bash
# Standard generation
mlx_vlm.generate \
  --model StargazerLabs/Qwen3.8-32B-Jumbo-3bit \
  --prompt "Your prompt here" --max-tokens 2048

# With MTP speculative decoding (use 4-bit drafter; no 3-bit drafter exists)
mlx_vlm.generate \
  --model StargazerLabs/Qwen3.8-32B-Jumbo-3bit \
  --draft-model mlx-community/Qwen3.8-27B-MTP-4bit \
  --prompt "Your prompt here" --max-tokens 2048

Related