Modelpublic

deepgrove/maple-preview-GGUF

sourceHugging Facemitupdated 1mo agoView on Hugging Face

90likes19kdownloads

Model Card

Maple-Preview-GGUFs

Custom llama.cpp fork and setup instructions: github.com/deepgrove-ai/llama.cpp

We include the following gguf variants:

Variant	GGUF size
TQ1_0 + Q4_K head	4.64 GiB
TQ1_0 + FP16 head	5.06 GiB
TQ2_0 + Q4_K head	5.50 GiB
TQ2_0 + FP16 head	5.91 GiB

TQ10 and TQ20 are different ternary packing schemes. Use TQ20 for generally faster speeds but slightly higher memory. LM-head is kept in higher precision - either Q4k or FP16.

Speed

M5 Pro, CPU-only, 16 threads, 512 prompt tokens, 128 generated tokens, 3 repetitions:

Matrix weights	LM head	GGUF size	Prefill (pp512)	Decode (tg128)
TQ1_0	FP16	5.06 GiB	515.41 ± 0.28 tokens/s	161.06 ± 0.57 tokens/s
TQ1_0	Q4_K	4.64 GiB	513.33 ± 3.62 tokens/s	231.13 ± 0.13 tokens/s
TQ2_0	FP16	5.91 GiB	618.57 ± 2.12 tokens/s	169.81 ± 2.94 tokens/s
TQ2_0	Q4_K	5.50 GiB	610.48 ± 3.76 tokens/s	252.74 ± 0.37 tokens/s

Architecture

Maple-Preview is a 20B-A1B reasoning model designed from the start for efficient on-device inference. It utilizes a 24-layer, 256-expert (8 active) configuration with 3:1 SWA-512:GA attention.

Evaluation

On benchmarks, Maple-Preview sets a new point on the Pareto frontier for both memory-to-performance and speed-to-performance, demonstrating its strong reasoning capabilities. However, we note that this preview is focused primarily on raw reasoning and, as such, may underperform on agentic benchmarks. We intend to continue improving general performance through extended training before Maple's full release.

License

Maple-Preview is released under the MIT License.