KCh3dRi4n/Qwen3-coder-23B-A3B-KREAP-MLX

Qwen3-Coder-23B-A3B-KREAP-MLX (25% routed pruning)

Korean-preserving one-shot MoE expert pruning of Qwen3-Coder-30B-A3B-Instruct, produced with **K-REAP**. Routed experts pruned 128 → 96 per layer (25%); ~23B total params, 3.3B active (top-8 unchanged). No fine-tuning.

Unlike vanilla REAP (English/code calibration), K-REAP detects and hard-protects the experts that carry Korean, so pruning preserves Korean without sacrificing English/coding. See K-Guard-REAP for the full study.

Korean loss minimized vs. standard REAP

The standard REAP recipe collapses Korean at comparable compression: Cerebras Qwen3-Coder-REAP-25B (19.5%, no language protection) drops Korean MC macro 79.3 → 56.6 (−22.7pp) with generation collapse 115/128. By hard-protecting the Korean expert path, this model keeps Korean MC macro at 77.4 (−1.9pp) with 0/128 collapse, i.e. the Korean loss is reduced to a few points or less. (K-REAP removes ~92% of REAP's Korean damage.)

Benchmarks (vs base Qwen3-Coder-30B; Δ = base delta, %p)

한국어 (Korean, self-harness)

English reasoning/knowledge (llm-evalbox)

Math

Coding

Safety/Bias

Macro

Usage (MLX)

from mlx_lm import load, generate
model, tok = load("KCh3dRi4n/Qwen3-coder-23B-A3B-KREAP-MLX")
print(generate(model, tok, prompt="한국어로 자기소개를 해줘.", max_tokens=256))

Standard HF safetensors (BF16) — also loadable with transformers. Coverage note: KoBEST & EvalPlus are full sets; KMMLU/evalbox/LiveCodeBench are deterministic subsets (seed 42). Effect sizes ≫ sampling CI. Details: K-Guard-REAP docs/10_MASTER_RESULTS.md §7.

Method

REAP saliency (router gate × expert-output L2 norm, conditional mean) restricted to Korean segments + Korean↔English contrast + rare/rollout protection → hard-protected survivor set → streaming safetensors surgery (uniform experts/layer). Framework: https://github.com/Chedrian07/K-REAP