Modelpublic

bknyaz/Qwen3-235B-A22B-Instruct-2507-REAM

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

1likes21downloads

Model Card

Qwen3-235B-A22B-Instruct-2507-REAM

This model is a compressed version of Qwen/Qwen3-235B-A22B-Instruct-2507. It is obtained by reducing the number of experts in each MoE layer from 128 to 96. This reduction is achieved by the REAM method described in https://bknyaz.github.io/blog/2026/moe/. The compressed model has 180B params (350GB) instead of 235B (470GB) of the original model, reducing storage and GPU memory requirements by roughly 25%. At the same time, the model retains >=97% of the original model's performance on a variety of benchmarks (see Results section below). Additional efficiency optimization (e.g., quantization) can be added similarly to the original model.

See additional details at Qwen3-30B-A3B-Instruct-2507-REAM.

Results

Model	IFeval	AIME25	GSM8K	GPQA-D	HumanEval	LiveCodeBench	AVG
Qwen3-235B-A22B-Instruct-2507	93.3	66.7	89.4	48.5	95.1	46.4	73.2
Qwen3-235B-A22B-Instruct-2507-REAM	90.4	63.3	88.2	44.4	94.5	49.5	71.7

License

Please refer to the license of the original model Qwen/Qwen3-235B-A22B-Instruct-2507.