Modelpublic

mavis-ai/Qwen3.6-35B-MoE-Q8

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes223downloads

Model Card

mavis-ai/Qwen3.6-35B-MoE-Q8

This repository contains an MLX-ready Q8 quantized build of Qwen3.6-35B-A3B, prepared for local inference on Apple Silicon Macs.

This is not a newly fine-tuned model. It is a local MLX redistribution converted from the official Qwen3.6-35B-A3B checkpoint. No additional training or architecture-level modification has been applied.

Source revision: 995ad96eacd98c81ed38be0c5b274b04031597b0.

Important Notice

This package uses a controlled, Gemma4-like mixed quantization policy. It is not a plain "quantize every linear layer to Q8" build.

For the original model card, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream model:

—Base model: <https://huggingface.co/Qwen/Qwen3.6-35B-A3B>

Quantization

This package uses MLX affine quantization with a mixed layout intended to preserve quality-sensitive tensor groups:

—Source checkpoint: Qwen/Qwen3.6-35B-A3B
—Default language tensor quantization: MLX affine Q8
—Token embeddings: BF16
—LM head: BF16
—MoE routing gates and shared expert gates: BF16
—Vision / multimodal components: BF16
—Group size: 64
—Mode: affine

The goal is to keep the model closer to the higher-precision routing and multimodal behavior expected from the original checkpoint while still reducing local storage and memory requirements. For this Q8 build, vocabulary-facing tensors are kept at BF16.

Model Notes

Qwen3.6-35B-A3B is a multimodal Mixture-of-Experts model with 35B total parameters and approximately 3B activated parameters. The local source checkpoint reports:

—Hidden size: 2048
—Layers: 40
—Experts: 256
—Activated experts: 8 routed experts plus shared expert
—Context length: 262144

Usage

Install or update the MLX runtime used for Qwen3.6 / multimodal models:

bash

pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet

Download the model:

bash

hf download mavis-ai/Qwen3.6-35B-MoE-Q8 \
  --local-dir ~/Models/mlx/Qwen3.6-35B-MoE-Q8

Run a quick text generation test:

bash

python -m mlx_vlm.generate \
  --model ~/Models/mlx/Qwen3.6-35B-MoE-Q8 \
  --max-tokens 128 \
  --temperature 0.0 \
  --prompt "Answer briefly: what can you do?"

Files

Recommended repository files:

text

.gitattributes
LICENSE
README.md
chat_template.jinja
config.json
configuration.json
generation_config.json
merges.txt
model-00001-of-00008.safetensors
model-00002-of-00008.safetensors
model-00003-of-00008.safetensors
model-00004-of-00008.safetensors
model-00005-of-00008.safetensors
model-00006-of-00008.safetensors
model-00007-of-00008.safetensors
model-00008-of-00008.safetensors
model.safetensors.index.json
preprocessor_config.json
processor_config.json
tokenizer.json
tokenizer_config.json
video_preprocessor_config.json
vocab.json

License

This repository redistributes a quantized derivative of Qwen3.6-35B-A3B, which is distributed under the Apache License 2.0. A copy of the license is included in the LICENSE file in this repository.

Modification Notice

Compared with the official Qwen source checkpoint, this repository applies the following packaging modification:

text

The source checkpoint was converted to MLX format and quantized with a controlled mixed Q8/BF16 policy for local MLX inference.

No fine-tuning, additional training, or architecture-level modification has been applied.