degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm

Escha Qwen 3.8 27b (W2 · Q8_0 head · dense) — AMD ROCm port

The Escha dense 27B model is a 2-bit-quantized build of the Qwen3.8-27B backbone (qwen35 architecture), converted to GGUF for llama.cpp and ported to run on AMD ROCm (gfx1100).

• —Backbone: Qwen/Qwen3.8-27B (Apache-2.0), via the Escha 2-bit quant `EschaLabs/Qwen3.8-27B-Escha-W2`
• —Format: Escha 2-bit (W2) weights + Q8_0 head; general.license = apache-2.0
• —Architecture: qwen35 (dense, non-MoE)
• —Size: 10.31 GB (9.6 GiB) on disk; 2054 tensors
• —Target: AMD Radeon RX 7900 XTX (gfx1100, 25.75 GB) — tested

Model configuration

Usage

Grab the model file (the only file in this repo):

huggingface-cli download degenmeowster/Escha-Qwen3.8-27B-W2-Q8E-ROCm \
  Escha-Qwen3.8-27B-W2-Q8E.gguf --local-dir .

Then build + serve through the ROCmFPX fork of llama.cpp (maintained under Chromadera). The escha decode kernel is a HIP kernel added to that tree on the escha-dense-27b branch.

git clone https://github.com/Chromadera/ROCmFPX
cd ROCmFPX && git checkout escha-dense-27b
scripts/build-rdna3.sh        # auto-selects gfx1100; see doc for ROCm-path overrides

cd build-rdna3/bin
./llama-server \
  -m ./Escha-Qwen3.8-27B-W2-Q8E.gguf \
  -ngl 99 -mg 0 -fa on --jinja \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  -c 204800 --no-warmup -np 1

Full build + run + sanity-check guidance: `docs/ESCHA-DENSE-27B-RUN-AMD.md`

Benchmarks

Measured via llama-bench / llama-perplexity, full offload, batch 1. Three columns: this port (llama.cpp GGUF, ROCm/gfx1100), Ajay's `aj9o9/Qwen3.8-27B-Escha-W2-GGUF` (llama.cpp GGUF, CUDA), and the Escha reference build (`EschaLabs/Qwen3.8-27B-Escha-W2`, native SGLang, RTX 4090).

Comparability: this port and Ajay's are both llama.cpp GGUF — directly comparable. Escha's figures are the native SGLang runtime on an RTX 4090, a different engine and card, so read that column as reference-scale, not an apples-to-apples run.

Speed

¹Escha prefill is quoted from their ISL÷TTFT at batch 1 (2,600–2,820 tok/s on a 4090); the figure here is the midpoint. Escha's decode is single-stream.

Decode holds against Ajay's 3090 (25.8 vs 24.03) — decode is the memory-bandwidth-bound figure and the escha decode path saturates bandwidth here. Prefill is lower (162 vs 700) because the escha dense prefill path is more prefill-bound on this GPU.

Quality

Quality protocol differs by provider, so this is not a single scoring matrix — each row is the metric's own best measured value. This port and Ajay run llama.cpp and report perplexity; Escha reports task accuracy measured through its native SGLang runtime.

Note on axes. Quality protocol differs by provider, so this is not a single scoring matrix — each row is that metric's own best measured value, and the meaning of "better" differs by row (perplexity lower is better; task accuracy higher is better). This port and Ajay run llama.cpp and report perplexity (loss) and determinism; Escha reports task accuracy measured through its native SGLang runtime. A lower perplexity and a higher task score are not on the same scale, so read them as complementary evidence rather than a head-to-head. Escha reports its figures without speculation and on the same build upstream; this port reproduces that build deterministically.

Context-capacity VRAM (q8_0 / q8_0, full lossless)

The port runs the full 262144 context on a 24+ GB card because only 16 of 64 layers are full attention (see KV-cache below).

Technical notes

Reasoning

The Qwen3.8-27B backbone is a thinking model. It emits <think>...</think> reasoning blocks that the llama.cpp server splits into reasoning_content / content. It works with a Qwen-style Jinja chat template and honors per-request reasoning_effort / reasoning_budget_tokens (e.g. the quimmedes `chat_template.jinja` via --chat-template-file), so you can drive thinking depth by effort level.

KV-cache

The model is a 48 linear + 16 full-attention Qwen35 hybrid (full_attention_interval = 4), so only ~16 layers carry a real KV cache. With q8_0 / q8_0 K/V this fits even at the full 262144 context (~20.7 GB). A lighter footprint is available via the fork's TurboQuant K/V types (--cache-type-v turbo3 / turbo4).

Quantization

Tensor breakdown (2054 tensors):

Credits

This build builds on prior OSS work:

• —Ajay — https://github.com/Ajay9o9/llama.cpp-escha (upstream escha decode path + tests), published at https://huggingface.co/aj9o9/Qwen3.8-27B-Escha-W2-GGUF
• —Escha team / EschaLabs — the Escha 2-bit codec + reference runtime (escha 1.2.1+qwen3dense)
• —charlie12345 — original ROCmFPX fork this port was based on (https://github.com/charlie12345/ROCmFPX); now maintained under Chromadera at https://github.com/Chromadera/ROCmFPX

License

Apache-2.0 (matching the Qwen3.8-27B base and the general.license field in the GGUF).