Tostibrown/Qwen3.6-35B-A3B-4bit-textonly

Qwen3.6-35B-A3B-4bit-textonly

A text-only derivative of `mlx-community/Qwen3.6-35B-A3B-4bit` with the vision tower removed. Produced for Mei, a narrow native Swift/MLX inference server for Apple Silicon, where the model is used purely for text/coding agent work and the vision weights are dead weight.

This is not a re-quantization. No weights were converted, re-quantized, or retrained. Tensor payloads for every retained tensor are bit-identical to the upstream checkpoint.

What changed

Removing vision_config is not cosmetic: in the MLX Swift stack it is the key that routes the bundle to the VLM model factory rather than the text LLM factory. With it gone, this bundle's config.json key set is a strict subset of the text-only Qwen3_5MoeForConditionalGeneration sibling checkpoints, and the language model loads directly.

The text tower is untouched: 40 layers, 256 experts, top-8 routing, moe_intermediate_size 512, full_attention_interval 4 (10 full-attention + 30 linear-attention layers), hidden size 2048, vocab 248320. Quantization is the upstream recipe unchanged — 4-bit affine, group size 64, with mlp.gate and mlp.shared_expert_gate held at 8-bit.

Capability removed, not disabled

Image and video input do not work with this checkpoint. The vision weights are gone, not switched off. If you need the VLM capability, use the upstream repository. There is no way to re-enable it here.

Verification

The conversion is verified, not asserted. conversion-provenance.json in this repository records the full detail. Summary:

• —Every retained tensor's payload was SHA-256 checked against the source as it was written, and independently re-checked afterwards on a random 50-tensor sample by raw file-offset digest: all identical.
• —The reference safetensors implementation opens all four shards and enumerates 1757 tensors with no errors.
• —All 333 removed tensors were confined to shard 1; shards 2–4 are byte-for-byte copies of the source shards.
• —The data segment start is padded to an 8-byte boundary and the rewritten shard has 0 unaligned tensors, so MLX's mmap loader maps it zero-copy rather than realigning the payload into anonymous RAM.

Usage

mei --model-dir <path-to-this-repo> \
    --served-model-id Tostibrown/Qwen3.6-35B-A3B-4bit-textonly \
    --port 8024 --context-cap 65536 --prefill-step-size 512

Also loadable by any MLX text-generation stack that supports qwen3_5_moe.

Sized for a 32 GB Apple Silicon machine: ~18.2 GiB of weights, leaving room for the KV cache and runtime overhead. The hybrid attention topology means the KV cache stays small at long context — only 10 of the 40 layers carry a conventional KV cache; the other 30 hold fixed-size recurrent state.

Provenance

• —Source: mlx-community/Qwen3.6-35B-A3B-4bit, revision 38740b847e4cb78f352aba30aa41c76e08e6eb46
• —Produced by strip_vision_tower.py (Mei project tooling), 2026-09-06

The local source copy had earlier been opened by MLX's safetensors storage healer, which added header padding in place while preserving the tensor payload exactly. So the file bytes here are not identical to upstream's files, but the tensor payloads are. See conversion-provenance.json.

Licence and attribution

Inherits the licence of the upstream model. All model weights are the work of the original Qwen authors and the mlx-community conversion; this repository contributes only the removal of the vision tower. It is not an official Qwen or mlx-community release and should not be described as one.