Modelpublic

YTan2000/Laguna-XS-2.1-TQ3_4S

sourceHugging Faceopenmdw-1.1updated 2mo agoView on Hugging Face

1likes62downloads

Model Card

TurboLaguna-XS

Canonical artifact: Laguna-XS-2.1-TQ3_4S

TurboLaguna-XS is the TurboQuant GGUF build of Poolside's Laguna XS 2.1 — a sigmoid-routed mixture-of-experts coding model with 256 experts per layer, a shared expert, QK-norm, and hybrid YaRN/sliding-window RoPE.

The exact file and runtime artifact name is:

—Laguna-XS-2.1-TQ3_4S.gguf

Required Runtime

This model uses the custom `TQ3_4S` tensor type. Stock llama.cpp builds cannot load it. You must use the TurboQuant runtime fork: [turbo-tan/llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3) This is a standard (non-MTP) model — no draft-MTP flags are needed.

Parent Model

—Upstream parent: poolside/Laguna-XS-2.1-GGUF
—Source quant: Laguna-XS-2.1-BF16.gguf (63.8 GB, 16.01 bpw)
—Format conversion and TurboQuant packaging: turbo-tan/llama.cpp-tq3

Files

File	Size	Notes
`Laguna-XS-2.1-TQ3_4S.gguf`	16 GB (4.05 bpw)	Main model — 678 tensors, 40 layers × 256 routed experts
`thumbnail.png`	—	Model card image
`benchmark.png`	—	Benchmark summary

Quantization Recipe

Quantized from the official BF16 GGUF using the standard TQ3_4S recipe:

bash

./build/bin/llama-quantize --allow-requantize \
  --output-tensor-type q6_K \
  --token-embedding-type q6_K \
  Laguna-XS-2.1-BF16.gguf \
  Laguna-XS-2.1-TQ3_4S.gguf \
  TQ3_4S

Tensor policy:

—Routed experts, attention projections, shared experts → tq3_4s (4.0 bpw)
—Token embeddings, output head → q6_K
—Norms, gates, biases → f32 (untouched)

Result: 63.8 GB → 16 GB (3.98× compression), 42% smaller than Poolside's own Q4KM (20 GB).

Recommended Runtime

bash

./build/bin/llama-server \
  -m Laguna-XS-2.1-TQ3_4S.gguf \
  --host 127.0.0.1 --port 8080 \
  -c 8192 -np 1 -ngl 99 -fa on \
  --reasoning off --jinja

Build note:

—-fa on is the runtime flash-attention flag, not the CMake GGML_CUDA_FA_ALL_QUANTS build flag.

GPU Memory Profiles

GPU memory	Suggested context	KV cache	Notes
16 GiB	`4096`	`-ctk q4_0 -ctv tq3_0`	Tight fit — keep context small
24 GiB	`8192` to `32768`	`-ctk q8_0 -ctv tq3_0`	Validated desktop profile
128 GiB GB10	`65536+`	`-ctk q4_0 -ctv tq3_0`	Full headroom for long context

Tested Hardware

—NVIDIA RTX 3090 24 GB — primary validation platform
—llama.cpp-tq3 fork, branch feat/laguna-arch (Laguna arch from upstream ggml-org/llama.cpp#25165)

Benchmarks

All scores: greedy decoding, reasoning off, -ngl 99 -fa on, RTX 3090.

Benchmark	Score	tok/s
HumanEval (base)	0.805	196
HumanEval+ (extra tests)	0.762	196
MBPP (base)	0.833	199
MBPP+ (extra tests)	0.720	199
Hard86 (20 tasks / 86 assertions)	64.0% (55/86)	202
BenchLoop coding	100.0 (12/12)	—
BenchLoop overall	73.7	—
BenchLoop speed	96.4 (9/9)	—

Comparison (all TQ3_4S, same RTX 3090)

Model	HE+	MBPP+	Hard86	Coding	tok/s	Size
Laguna XS 2.1	0.762	0.720	64.0%	100.0	196	16 GB
Qwen3.5 9B	0.671	0.563	44.2%	79.2	134	4.5 GB
Qwen3.6 27B MTP	0.927	0.878	—	100.0	42–54	12.9 GB

Laguna XS is a coding specialist: perfect BenchLoop coding (same as the 27B), +19.8pp Hard86 over the 9B, at 3.6–4.6× the 27B's decode speed.

Validation

llama-simple-chat coherence smoke: PASS
llama-server --reasoning off strict smoke: PASS (content = "ok")
llama-bench pp2048: 745 tok/s
llama-bench tg128: 196 tok/s
evalplus HE/HE+/MBPP/MBPP+: scored (see above)
hard86: 55/86
benchloop v0.2.3: overall 73.7

License

—Parent model: OpenMDW-1.1 (Poolside)
—Runtime: turbo-tan/llama.cpp-tq3 (MIT)