cesarsal1nas/Qwen3.8-27B-EXL3-GGUF

Qwen3.8-27B EXL3 GGUF

llama.cpp GGUFs of GestaltLabs/Qwen3.8-27B-EXL3-11.5GB (packed EXL3 of Qwen/Qwen3.8-27B).

Converted with llama.cpp-exl3xe2cuda (convert_hf_to_gguf.py). The Viterbi encode was ExLlamaV3 on CUDA. v3 GGUF convert is CPU; xe2-16 rematerialize is not. This repo is so a box without NVIDIA can still get both files.

Not affiliated with ggml-org, Turboderp, GestaltLabs, or Qwen. Not an upstream llama.cpp PR.

Files

Quality default is v3. Same ~2.87 bpw, MTP included.

• —Qwen3.8-27B-EXL3-11.5GB-text-mtp-v3.gguf pack v3 MMA. Loads CUDA (sm_86/89) and Intel Xe2 SYCL. wiki.test.raw 16x2048 PPL 6.42 (CUDA).

• —Qwen3.8-27B-EXL3-11.5GB-text-mtp-xe2-16.gguf pack xe2-16 (K-column). Intel Xe2 SYCL only. CUDA GGML_ABORTs this file. Rematerialize of the GestaltLabs v3 tiles, not a new BF16 encode. wiki.test.raw 16x2048 PPL 6.62.

Need that fork to load either GGUF. Stock llama.cpp has no EXL3 type.

Run

NVIDIA, v3:

./build-cuda/bin/llama-server --device CUDA0 \
  -m Qwen3.8-27B-EXL3-11.5GB-text-mtp-v3.gguf \
  -ngl 99 --flash-attn on --port 8080

Intel Arc B65, xe2-16 (spare card is level_zero:1 on a dual-B65 box):

ONEAPI_DEVICE_SELECTOR=level_zero:1 ./build-sycl/bin/llama-server \
  --device SYCL0 -m Qwen3.8-27B-EXL3-11.5GB-text-mtp-xe2-16.gguf \
  -ngl 99 --flash-attn on --port 8080

Numbers and flags: README on GitHub.

Do not run ExLlama convert.py EXL3_PACK=xe2-16 from dense BF16 expecting v3 PPL.

Credits

• —Base model: Qwen Team, Qwen/Qwen3.8-27B, Apache-2.0
• —Packed EXL3 safetensors: GestaltLabs/Qwen3.8-27B-EXL3-11.5GB, Apache-2.0
• —EXL3 format and CUDA kernels: Turboderp / ExLlamaV3, MIT
• —llama.cpp / ggml: The ggml authors, MIT
• —GGUF convert, SYCL EXL3, xe2-16 rematerialize: llama.cpp-exl3xe2cuda

Keep those notices. This card does not relicense anyone else's work.