Modelpublic

WaveCut/Qwen3.5-2B-MLX-DWQ-4bit

sourceHugging Faceapache-2.0updated 9h agoView on Hugging Face

0likes7downloads

Model Card

Qwen3.5-2B — MLX DWQ 4-bit

Qwen3.5-2B for Apple silicon in MLX format. The language backbone uses 4-bit affine DWQ with selected 8-bit matrices, group size 32. The vision tower retains the original BF16 weights. Weight file: 1.90 GB.

Compatible with unmodified mlx-lm==0.31.3 and mlx-vlm==0.7.3, tested on stable mlx==0.32.2.

Quality

DWQ calibration uses multilingual natural text from FineWeb, FineWeb2 and Wikipedia, balanced across 16 languages. Source revision: 15852e8c16360a2fea060d615a32b45270f8a8fc.

On 65,408 held-out tokens across 16 languages, teacher KL is 21.4% lower than the 4-bit DWQ baseline without selective 8-bit matrices:

Model	KL ↓	Perplexity ↓
DWQ 4-bit baseline	0.08339	19.8703
This model	0.06554	19.6672

KL is measured against the BF16 source at temperature 1. These text metrics do not establish writing, instruction-following or vision quality. Image input passed a pipeline smoke test. Evaluation details.

Use

bash

python -m pip install "mlx-vlm==0.7.3"
mlx_vlm.generate --model WaveCut/Qwen3.5-2B-MLX-DWQ-4bit \
  --image /path/to/image.jpg --prompt "Describe this image." \
  --max-tokens 128 --temperature 0