Modelpublic

OsaurusAI/Qwen3.6-27B-JANG_4M

sourceHugging Faceapache-2.0updated 26d agoView on Hugging Face

2likes214downloads

Model Card

<h3 align="center">Qwen 3.6 27B — JANG4M (MLX)</h3> <p align="center">Balanced 4-bit quantization of Alibaba's hybrid linear/full-attention dense 27B VL — full-attention <code>q/k/v/o</code>, embeddings and <code>lmhead</code> at 8-bit affine; dense FFN, linear-attention projections and vision tower at 4-bit affine.</p>

Model Details

Property	Value
Base model	`Qwen/Qwen3.6-27B`
Parameters	27.3 B, dense (no MoE)
Architecture	`qwen3_5` — 64 decoder layers: 48 `Gated DeltaNet` (linear-attn) + 16 full-attention with `swish` output gate
Quantization	JANG_4M — mixed 4/8-bit native MLX affine
Package size on disk	17.5 GB across 11 shards
Avg bits/param	4.45
vs BF16 source	52 GB → 17.5 GB, 3.0× compression
Context	262 144 native; upstream card reports up to ~1 M with YaRN
Vision tower	27-layer ViT (hidden 1152, patch 16), temporal_patch 2, quantized at 4-bit with patch-embed axes pre-transposed to MLX layout
Chat format	Qwen `im_start`/`im_end` with `enable_thinking` toggle

JANG_4M bit allocation

Category	Bits	Group	Notes
Full-attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`) — 16 layers	8	64	Precision-critical. `q_proj` is fused with a swish output gate (half queries / half gate)
Embedding (`embed_tokens`), `lm_head`	8	64	Input/output precision bound
Dense FFN (`mlp.gate_proj`, `mlp.up_proj`, `mlp.down_proj`) — 64 layers	4	64	Bulk of parameters
Linear-attention projections (`in_proj_qkv`, `in_proj_z`, `in_proj_b`, `in_proj_a`, `out_proj`) — 48 layers	4	64
Vision tower (27 ViT layers)	4	64
Norms, `A_log`, `dt_bias`, `conv1d`	bf16	—	Passthrough
MTP head	—	—	Stripped — mlx_vlm doesn't use it

Per-module bit overrides are encoded directly in config.json["quantization"] (65 overrides for the 8-bit tier), so any MLX-compatible runtime can load the bundle without custom decode paths.

Why mixed 4/8? This dense 27B has 16 full-attention layers whose q_proj is fused with a sigmoid(gate) multiplier — activation noise near the gate transition zone is amplifying if attention is 4-bit. JANG_4M keeps those projections at 8-bit while absorbing the compression into the 64-layer dense FFN.

Architecture notes

—Hybrid attention stack: 48 of 64 layers use Gated DeltaNet, a linear-attention / delta-rule hybrid with a grouped conv1d input path and per-head A_log / dt_bias state — constant memory in sequence length. The other 16 layers (one every 4, given by full_attention_interval: 4) use full softmax attention with `attn_output_gate: true` — q_proj produces a fused (queries, gate) tensor; attention output is multiplied by sigmoid(gate) before o_proj.
—Partial rotary embeddings: only the first 25% of head dim rotates (partial_rotary_factor: 0.25), rope_theta = 1e7. Position metadata for mixed text/image/video (mrope_section, mrope_interleaved: true) is preserved in config.json.
—Dense FFN: no MoE. Each layer has gate_proj/up_proj (5120 → 17408) + down_proj (17408 → 5120) with SwiGLU activation.
—Vision tower: qwen3_vl ViT, 27 layers, hidden 1152, patch 16, temporal_patch 2. Produces video token sequences via 3D conv patch-embed (pairs of frames merge into one temporal patch).

Usage

Load in **Osaurus** on Apple Silicon (macOS) — single-click deploy, local chat + vision, no Python setup. The bundle also loads in any Apple Silicon MLX runtime that supports qwen3_5 VL bundles with per-module quantization config (see config.json["quantization"]).

Reasoning on/off, image inference, and video inference are all verified on this quant.

Verified modalities

Test	Result
Chat template (with + without thinking)	✓ coherent
Text (enable_thinking=False): "The capital of France is" → "Paris"	✓
Text: "Translate to French: Hello, how are you?" → "Bonjour, comment allez-vous ?"	✓
Text: `def fibonacci(n):` → correct recursive continuation	✓
VL image: solid red/green/blue/yellow 224×224 → correct color ID	✓ 4/4
VL video: 4-frame RGBY sequence → structurally coherent description	✓

The 4-frame RGBY video encodes as 2 temporal patches via temporal_patch_size=2, which the model perceives as a 2-region color composition — identical behavior to the BF16 source. This is not a quant artifact.

For synthetic PIL-frame video tests, set processor.video_processor.do_sample_frames = False so each frame maps 1:1 to a patch.

MMLU-200 (10 subjects × 20 questions, reasoning OFF)

Both quants evaluated on the same 200-question slice of MMLU with enable_thinking=False (direct answer, no <think> preamble). Same prompts, same greedy decode, same extraction.

Subject	MXFP4	JANG_4M	Δ (JANG − MXFP4)
abstract_algebra	12/20 (60.0%)	15/20 (75.0%)	+3
anatomy	18/20 (90.0%)	16/20 (80.0%)	-2
astronomy	20/20 (100.0%)	19/20 (95.0%)	-1
collegecomputerscience	16/20 (80.0%)	16/20 (80.0%)	0
college_physics	15/20 (75.0%)	15/20 (75.0%)	0
highschoolbiology	19/20 (95.0%)	19/20 (95.0%)	0
highschoolchemistry	16/20 (80.0%)	15/20 (75.0%)	-1
highschoolmathematics	12/20 (60.0%)	14/20 (70.0%)	+2
logical_fallacies	20/20 (100.0%)	19/20 (95.0%)	-1
world_religions	19/20 (95.0%)	17/20 (85.0%)	-2
Total	167/200 (83.5%)	165/200 (82.5%)	−1.0 pp

Both quants are strong baselines on reasoning-OFF MMLU. MXFP4 edges ahead by 1 pp overall. JANG4M wins on the harder math-heavy subjects (`abstractalgebra +3, highschoolmathematics +2) — plausibly because the 8-bit full-attention projections carry more signal on multi-step symbolic chains. MXFP4 wins on rote-recall subjects (anatomy, world_religions`) by ~2 each, closer to ties on factual/scientific subjects.

Reasoning ON: not yet measured. Qwen 3.6 is a reasoning-optional model — with enable_thinking=True the model generates a <think>…</think> block before answering, which typically lifts MMLU significantly. Reasoning-ON benchmarks for both quants are planned as a follow-up.

Hardware notes

17.5 GB weights on disk; once loaded, expect ~18–22 GB resident plus KV cache.

Mac	Works?	Notes
24 GB unified	⚠️	Text + image tight; no video
32 GB unified	✅	Comfortable for text + image + short video
48 GB+ unified	✅	Full context + VL + video

License

Apache 2.0 — inherits from the base model.