summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

LLama.cpp unsported

https://github.com/vnlpscale/llama.cpp sported

Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

GGUF Q4_K_M quantization of [summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64](https://huggingface.co/summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64).

Model Details

• —Base Model: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64
• —Architecture: Qwen3.8 / qwen35
• —Parameters: ~27B
• —Quantization: Q4_K_M
• —Format: GGUF
• —Maximum Context Length: 262,144 tokens
• —License: Apache-2.0

Download

Using the Hugging Face CLI:

huggingface-cli download \
  summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

With newer versions of the Hugging Face CLI, you can also use:

hf download \
  summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

llama.cpp

Use a recent build of llama.cpp.

CLI

./llama-cli \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

Server

./llama-server \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

-ngl 99 attempts to offload all model layers to the GPU. Reduce this value if the model does not fit in available VRAM.

Quantization

This repository provides the Q4_K_M quantization.

Q4_K_M provides a practical balance between model quality, memory usage, and inference performance for a model of this size.

The model was quantized from the higher-precision GGUF using llama-quantize:

llama-quantize \
  SpeedX27-GDN64-BF16.gguf \
  SpeedX27-GDN64-Q4_K_M.gguf \
  Q4_K_M

Context Length

The GGUF metadata specifies a maximum context length of 262,144 tokens.

You do not need to allocate the full context length during inference. Context size has a substantial effect on KV-cache memory usage.

For example:

# 8K context
-c 8192

# 32K context
-c 32768

# 64K context
-c 65536

Choose the context size according to available RAM/VRAM and your workload.

Compatibility

This model uses a modified Qwen architecture with GDN/recurrent components.

A recent version of llama.cpp is recommended. Older builds may not recognize the architecture or may lack support required by the model.

If loading fails, update llama.cpp to the latest revision before troubleshooting the GGUF itself.

Source Model

• —Original model: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64
• —GGUF repository: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

License

Apache-2.0.

See the original model repository for additional model information and any applicable usage notes.