quimmedes/Qwen3.8-Flash-Next-MTP-GGUF

Qwen 3.8 Flash Next - MTP Draft Speculative Model (GGUF)

This repository provides working MTP (Multi-Token Prediction) draft models in GGUF format for Qwen 3.8 Flash Next (and experimental architectures with hyper-connections / QSA / Hadamard KV rotations), powered by cafe-llama.cpp.

Compatible Engine

This MTP model requires the fork implementation supporting hyper-connection hidden states, QSA Hadamard KV rotation, and MTP layer graph generation:

• —Repository: https://github.com/quimmedes/cafe-llama.cpp

• —``` git clone https://github.com/quimmedes/cafe-llama.cpp.git

• —```

Building from Source

1. 1.NVIDIA CUDA (Windows / Linux)

CMake configure with CUDA backend

cmake -B build -DGGML_CUDA=ON

Build Release

cmake --build build --config Release -j 2

1. 1.Vulkan (Cross-Platform AMD / Intel / NVIDIA)

Requires Vulkan SDK installed

cmake -B build -DGGML_VULKAN=ON cmake --build build --config Release -j 2

1. 1.AMD ROCm / HIP (Linux / Windows) cmake -B build -DGGMLHIP=ON -DAMDGPUTARGETS="gfx1100;gfx1030" cmake --build build --config Release -j 2
2. 2.Apple Metal (macOS) cmake -B build -DGGML_METAL=ON cmake --build build --config Release -j 2
3. 3.CPU Only (AVX2 / AVX-512) cmake -B build -DGGMLCUDA=OFF -DGGMLVULKAN=OFF cmake --build build --config Release -j 2 -j N : The number of CPU threads the processor will use to compile, a safe number is the amount of physical cores of the processor.

Available Files

• —mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf: Quantized Q4KM MTP draft model (~2.65 GB) - Recommended
• —mtp-Qwen3.8-Flash-Next-Q6_K.gguf: Quantized Q6_K MTP draft model (~3.24 GB)
• —mtp-Qwen3.8-Flash-Next-Q8_0.gguf: Quantized Q8_0 MTP draft model (~3.94 GB)
• —mtp-Qwen3.8-Flash-Next-BF16.gguf: Full precision BF16 MTP draft model (~7.40 GB)

Usage with llama.cpp / llama-server

Run llama-server or llama-cli with --spec-type draft-mtp and point -md to your preferred MTP file. Setting --spec-draft-n-max 2 provides optimal acceptance rate (~50-70%) for a 1-layer MTP draft head:

llama-server   -m Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf   -md mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf   --spec-type draft-mtp   --spec-draft-n-max 2   -ngl 999   -hmoe   -fa on   -ctk q8_0 -ctv q8_0 -kvu   -c 8192 -b 1024 -ub 128

CLI Example:

llama-cli   -m Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf   -md mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf   --spec-type draft-mtp   --spec-draft-n-max 2   -ngl 999   -hmoe   -fa on   -ctk q8_0 -ctv q8_0   -p "Explain quantum entanglement simply."