GOBA-AI-Labs/PrunedHub-Qwen3-30B-A3B-EN-80pct-MxMoE

Qwen3-30B-A3B EN 80% MxMoE: Language-Aware Expert Pruning + Mixed Quantization

13.45 GB | 102/128 experts | Mixed Q5K/Q4K/Q3K | -22% size, -4pp MMLU | GGUF | Apache 2.0

A compressed variant of Qwen3-30B-A3B combining two techniques:

1. 1.Language-aware expert pruning (80% keep): calibration-based importance scoring with EN-optimized expert selection
2. 2.MxMoE mixed quantization: layer-level adaptive precision (Q5K for critical layers, Q4K for standard, Q3K for redundant)

Optimized for English tasks. See JP variant for Japanese-optimized version.

Highlights

• —13.45 GB: fits in 16 GB RAM for full GPU-resident inference
• —22% smaller than original Q4KM (17.28 GB)
• —MMLU 70% (no-think) / think-ON available with Qwen3 reasoning mode
• —GSM8K 94% (no-think, 50Q) -- math reasoning fully preserved
• —~57 tok/s on Apple M4 Pro 24GB with llama-server (Metal GPU)
• —Compatible with llama.cpp (build 7970+) and moe-stream

Benchmark Results

Model Details

How to Use

With llama.cpp (recommended)

# Run with llama-server (Metal GPU, ~57 tok/s)
llama-server \
  -m Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf \
  --port 8090 \
  -ngl 99 \
  -c 4096

With moe-stream (alternative)

Also compatible with moe-stream for MoE-optimized inference:

git clone https://github.com/GOBA-AI-Labs/moe-stream
cd moe-stream && cargo build --release --features accelerate

./target/release/moe-stream-server \
  --model Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf \
  --port 11434

With OpenAI-compatible API

Once llama-server or moe-stream-server is running:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="none")

response = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "Explain quantum entanglement in simple terms."}],
    max_tokens=512,
    temperature=0.7,
)
print(response.choices[0].message.content)

Requirements

• —[llama.cpp](https://github.com/ggerganov/llama.cpp) (build 7970+) or [moe-stream](https://github.com/GOBA-AI-Labs/moe-stream)
• —16 GB RAM minimum (model uses ~13.5 GB; rest for KV cache)
• —Apple Silicon (Metal) recommended; CUDA also supported

Compression Methodology

Step 1: Language-Aware Expert Pruning (80% Keep)

Each expert is scored using calibration-based importance, computed from actual inference on representative English text:

importance(l, e) = activation_magnitude(l, e) × routing_frequency(l, e)

The bottom 20% of experts per layer are removed (128 → 102 experts/layer). This model uses English-calibrated importance scores, meaning experts are selected to maximize English task performance.

Step 2: MxMoE Layer-Level Mixed Quantization

After pruning, each layer's expert FFN tensors are quantized at different precision levels based on layer importance:

Constraints:

• —Attention layers (12 total) are guaranteed Tier 2 or above
• —Layer importance = mean_expert_importance × mean_routing_frequency

Size Breakdown

Original Q4_K_M:      17.28 GB (128 experts × 48 layers)
After 80% pruning:    ~15.0 GB (102 experts × 48 layers)
After MxMoE:          13.45 GB (Q5K/Q4K/Q3K mixed)
Total reduction:      -22.2%

Pruning Curve (Qwen3-30B-A3B)

\* With MxMoE; 80% pruning alone = ~15.0 GB at Q4KM uniform.

Limitations

• —English-optimized: Pruning was calibrated on English text. For Japanese or multilingual use, see the JP variant
• —No fine-tuning: Pruned and requantized without any post-pruning training
• —STEM sensitivity: STEM accuracy (57.5%) is more affected by compression than Humanities (70%) or Social Sciences (86.7%)

Reproducibility

The pruning scripts and plans are available at GOBA-AI-Labs/moe-prune:

# 1. Download Qwen3-30B-A3B Q4_K_M GGUF

# 2. Generate importance scores
python scripts/qwen3_30b_importance.py

# 3. Apply language-aware pruning (80% keep, EN-calibrated)
python scripts/reprune_gguf.py \
  Qwen3-30B-A3B-Q4_K_M.gguf \
  Qwen3-30B-A3B-pruned80-EN.gguf \
  results/calibration/qwen3_30b_a3b/plan_80pct_en_natural.json

# 4. Apply MxMoE mixed quantization
llama-quantize --allow-requantize \
  --tensor-type-file results/mxmoe/tensor_types_pruned.txt \
  Qwen3-30B-A3B-pruned80-EN.gguf \
  Qwen3-30B-A3B-pruned80-EN-MxMoE.gguf q4_k_m

Citation

@misc{goba2026moe,
  title   = {Language-Aware Expert Pruning with Mixed Quantization for MoE Language Models},
  author  = {GOBA-AI-Labs},
  year    = {2026},
  url     = {https://github.com/GOBA-AI-Labs/moe-prune},
  note    = {Calibration-based pruning + MxMoE achieves 22\% size reduction on Qwen3-30B-A3B}
}

Acknowledgments

• —Qwen Team for releasing Qwen3-30B-A3B under Apache 2.0
• —llama.cpp for the GGUF format and llama-quantize tool
• —moe-stream for the MoE inference engine

License

This model inherits the Apache 2.0 License from the base model (Qwen/Qwen3-30B-A3B).