abhisheksharma0994/apple-m5-pro-24gb-llm-tps-benchmarks

Apple M5 Pro (24GB) — Local LLM tokens/sec benchmarks

Real-world tokens/sec numbers for 12 local coding/agentic LLMs, measured on a specific, commonly-owned but previously unbenchmarked configuration: Apple M5 Pro, 24GB unified memory. At the time of writing, no public benchmark existed for this exact chip + RAM combination for these models. Includes both the official newly-open-sourced Qwen3.8-27B and a popular uncensored/abliterated finetune of it, for comparison.

Hardware

• —Chip: Apple M5 Pro (15-core: 5 Efficiency + 10 Performance)
• —Unified memory: 24 GB
• —OS: macOS 26.6.2 (build 25G83)
• —GPU memory ceiling: default macOS iogpu.wired_limit_mb is 0 (auto, ~18GB on this machine); some models below required raising it to 22000 (22GB) to load at all — noted per-row.

Methodology

• —Backend: MLX (mlx_lm.generate) for MLX-format models, llama.cpp (llama-bench + llama-cli) for the one GGUF model.
• —Standard metric (std_prompt_tps / std_gen_tps): a short synthetic prompt ("Write a one-paragraph summary of how binary search works."), capped at 128 output tokens — directly comparable to the pp/tg convention already used across HF model cards and llama.cpp benchmark threads.
• —Real-task metric (codingtask_*): a fixed, non-trivial coding prompt — implement a thread-safe concurrent restaurant waitlist system as a FastAPI REST API — capped at 1500 output tokens. This measures throughput under a realistic agentic-coding workload, not just a synthetic string.
• —Single run per model, temperature 0, no warm-up discard. Every model was benchmarked with all background downloads finished and Chrome/Safari closed, to avoid contention skewing the numbers (an earlier download-contaminated pass is kept separately in results_with_downloads_active.csv in this repo for reference — it shows a real, measurable effect from background downloads on tok/s).
• —Correctness was checked, not just speed — the notes column flags any model whose coding-task output was garbage (repetition loops, unfinished reasoning) despite a valid tok/s number. A high tok/s on broken output is not a useful result.

Results

Full data with all columns (peak memory, standard-prompt numbers, exact notes) is in results.csv.

Key findings

• —Quantization level affects correctness, not just speed. Qwen3-Coder-30B-A3B's 3-bit quant produced correct code while the 4-bit quant of the same model degenerated into a repetition loop on the identical prompt — a smaller/lower-precision file was the more usable one here.
• —24GB has a hard ceiling around ~20GB models, independent of macOS GPU memory tuning. Qwen3.6-35B-A3B (23GB on disk) could not be coaxed into running even after raising iogpu.wired_limit_mb to 22000 — it caused severe system-wide memory thrashing instead of a clean out-of-memory error.
• —Background activity (downloads) measurably affects tok/s — see results_with_downloads_active.csv for the same models benchmarked while another model was still downloading in the background.
• —MoE models are not automatically fast on this hardware. Despite having only ~3B active parameters, Qwen3-Coder-30B-A3B's total 30B footprint still puts real memory pressure on a 24GB machine, and Qwen3.6-35B-A3B (35B total) doesn't run at all.
• —Newer ≠ better for this task. Qwen3.8-27B (both official and uncensored) is a reasoning-first model that spent its entire 1500-token budget on chain-of-thought planning and never emitted actual code — despite being the newest model in this set, older non-reasoning models like DeepSeek-Coder-V2-Lite completed the task correctly and faster.
• —`llama.cpp`'s default `--ctx-size 0` (load full model context) can silently cause a Metal out-of-memory error on large models even when the model weights themselves fit comfortably — it allocates KV cache for the model's full trained context length regardless of actual prompt length. Explicitly bounding context (-c 4096 here) fixed it.

Files

• —results.csv — the clean, final benchmark pass (all downloads finished, no background contention). This is the authoritative data.
• —results_with_downloads_active.csv — an earlier pass on the same models, run while other downloads were active in the background. Kept for comparison, not the primary result.