symrex/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-dequantized-oQ8e-mtp
3548
Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-dequantized-oQ8e-mtp
This model was quantized using oQ (oMLX v0.5.7) mixed-precision quantization.
Quantization details
- Model type: qwen35moe
- Bits: 8
- Group size: 64
- Format: MLX safetensors
Performance Benchmark
- Run on: Apple Mac Studio M4 Max 128GB
- https://omlx.ai/benchmarks/jscg0oge
oMLX - LLM inference, optimized for your Mac
https://github.com/jundot/omlx
Benchmark Model: Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-dequantized-oQ8e-mtp
Engine: Auto
Context: Code (Python)
================================================================================
Single Request Results
--------------------------------------------------------------------------------
Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem
pp1024/tg128 593.2 11.45 1726.2 tok/s 88.0 tok/s 2.055 560.7 tok/s 35.47 GB
pp4096/tg128 2304.0 11.72 1777.8 tok/s 86.0 tok/s 3.776 1118.0 tok/s 36.35 GB
pp8192/tg128 4822.6 14.61 1698.7 tok/s 71.2 tok/s 5.207 1578.3 tok/s 36.80 GB
pp16384/tg128 10594.2 12.62 1546.5 tok/s 79.9 tok/s 12.205 1352.9 tok/s 37.69 GB
pp32768/tg128 24743.9 13.66 1324.3 tok/s 73.8 tok/s 26.488 1241.9 tok/s 39.49 GB
pp65536/tg128 63873.6 15.89 1026.0 tok/s 63.4 tok/s 65.901 996.4 tok/s 43.07 GB
pp131072/tg128 193797.5 19.84 676.3 tok/s 50.8 tok/s 196.330 668.3 tok/s 50.36 GB
pp200000/tg128 408283.1 24.26 489.9 tok/s 41.5 tok/s 411.381 486.5 tok/s 58.03 GB
Continuous Batching
pp1024 / tg128
--------------------------------------------------------------------------------
Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)
1x 88.0 tok/s 1.00x 1726.2 tok/s 1726.2 tok/s 593.2 2.055
2x 147.4 tok/s 1.68x 654.8 tok/s 327.4 tok/s 3127.6 4.865
4x 221.4 tok/s 2.52x 1492.9 tok/s 373.2 tok/s 2631.8 5.056
8x 303.3 tok/s 3.45x 1477.5 tok/s 184.7 tok/s 5172.4 8.920Intelligence Benchmark Comparison
Intelligence Benchmark Comparison
Mode Sampled Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-dequantized-oQ8e-mtp
--------------------------------------------------------------------------------------------------------
MMLU Sample 1000/14042 88.3%
MMLU_PRO Sample 300/12032 81.3%
TRUTHFULQA Full 817 81.8%
HUMANEVAL Full 164 95.7%
LIVECODEBENCH Sample 300/1055 39.0%
--- Detail ---
Model: Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-dequantized-oQ8e-mtp
Benchmark Accuracy Correct Total Time(s) Think
--------------------------------------------------------------
MMLU 88.3% 883 1000 4470.9 Yes
MMLU_PRO 81.3% 244 300 2818.9 Yes
TRUTHFULQA 81.8% 668 817 3162 Yes
HUMANEVAL 95.7% 157 164 1495.8 Yes
LIVECODEBENCH 39.0% 117 300 14379 Yes