bullerwins/Laguna-S-2.1-AutoRound-W4A16

Laguna S 2.1 AutoRound W4A16 G16

This is a community AutoRound quantization of `poolside/Laguna-S-2.1`, selected as the best balanced checkpoint from a controlled INT4 size/quality study. Only the routed experts are quantized: their weights use symmetric INT4 with a 16-element group size, while attention, routers, the shared experts, embeddings, and LM head remain BF16.

Quantization and research summary

• —Method: AutoRound SignRound, W4A16, symmetric G16, 20 iterations, 64 calibration samples of 1,024 tokens, seed 42, algorithm extensions
• —Calibration: disjoint text from NeelNanda/pile-10k; evaluation prompts and reference logits were never used for calibration or layer selection
• —Active checkpoint size: 76.887 GiB, counting only index-referenced weight shards
• —Development KLD: 0.24832–0.25264
• —Disjoint holdout KLD: 0.25526–0.26197

KLD is measured as mean KL(BF16 reference || quantized model) over fixed 1,024-token windows from held-out WikiText-2, GSM8K, and MBPP text; lower is better. We explored group sizes from G128 through G8, RTN and SignRound, expert-only versus all-linear quantization, and architecture-motivated BF16 exceptions. Seven load-verified, mathematically non-dominated checkpoints were retained. G8 achieved the lowest deterministic development KLD (0.241321) at 93.411 GiB, while this G16 checkpoint was selected as the practical default because it saves 16.52 GiB and has comparable monitored holdout fidelity.

G16/G32/G64 measurements use vLLM's Humming WNA16 backend. Fresh-engine noise was approximately 0.004 KLD, so small differences inside the reported ranges should not be treated as meaningful.

<p align="center"> <img alt="Laguna S 2.1 AutoRound KLD versus active checkpoint size" src="lagunakldsize_frontier.png" width="1200px"> </p>

The chart shows the retained size/KLD frontier and the Poolside NVFP4 and FP8 reference checkpoints. Vertical bars represent observed fresh-engine ranges.

Original model README

The original Poolside model card is reproduced below.

<p align="center"> <img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-s-2-1-banner.svg" width="800px"> </p>

<p align="center"> <a href="https://openrouter.ai/poolside/laguna-s-2.1"><strong>Use on OpenRouter</strong></a> · <a href="https://vercel.com/ai-gateway/models/laguna-s-2.1"><strong>Use on Vercel AI Gateway</strong></a> · <a href="https://poolside.ai/blog/introducing-laguna-s-2-1"><strong>Release blog post</strong></a> </p>

<br>

Laguna S 2.1

Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated parameters per token, designed for agentic coding and long-horizon work. It sits between Laguna XS 2.1 (33B-A3B) and Laguna M.1 (225B-A23B) in the Laguna series and shares the family recipe: a token-choice router with softplus gating over 256 routed experts plus one shared expert, grouped-query attention, and interleaved full/sliding-window attention.

Highlights

• —Mixed SWA and global attention layout: 48 layers in a 1:3 global-to-SWA ratio (12 global attention layers, 36 sliding-window layers, window 512), with softplus attention gating and per-layer-type rotary scales
• —1M context: 1,048,576-token context window
• —Native reasoning support: interleaved thinking between tool calls, with per-request control via enable_thinking
• —Speculative decoding: a trained DFlash draft model is available for lower-latency serving
• —Quantized variants: FP8, NVFP4, INT4 and GGUF
• —OpenMDW-1.1 license: Use and modify the model and associated materials freely for commercial and non-commercial purposes (learn more about OpenMDW)

Model overview

• —Number of parameters: 118B total, ~8B activated per token
• —Layers: 48 (12 global attention, 36 sliding-window attention)
• —Experts: 256 routed (top-10) plus 1 shared expert
• —Attention: grouped-query, 8 KV heads, head dim 128; per-head softplus output gating
• —Sliding window: 512 tokens
• —Context window: 1,048,576 tokens
• —Vocabulary: 100,352 tokens (Laguna family tokenizer)
• —Modality: text-to-text
• —Reasoning: interleaved thinking with preserved thinking

Benchmark results

<p align="center"> <img alt="benchmarks" src="https://poolside.ai/assets/laguna/laguna-s-2-1-chart.svg" width="800px"> </p>

Benchmarks as of 21 July 2026. Laguna S 2.1 in bold; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI's official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories: trajectories.poolside.ai.

Usage

Laguna S 2.1 uses the same laguna architecture as Laguna XS 2.1, so the same engine integrations apply (vLLM, SGLang, Transformers, TRT-LLM, llama.cpp). At 118B parameters the BF16 checkpoint needs multiple GPUs (roughly 236GB of weights); quantized variants reduce this substantially.

vLLM

vllm serve \
    --model poolside/Laguna-S-2.1 \
    --tensor-parallel-size 4 \
    --tool-call-parser poolside_v1 \
    --reasoning-parser poolside_v1 \
    --enable-auto-tool-choice \
    --served-model-name laguna \
    --default-chat-template-kwargs '{"enable_thinking": true}'

[!NOTE] Optional: speculative decoding with DFlash. Pair with the Laguna S 2.1 DFlash draft model by adding --speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":7,"method":"dflash"}'.

SGLang

python -m sglang.launch_server \
  --model-path poolside/Laguna-S-2.1 \
  --tp-size 4 \
  --reasoning-parser poolside_v1 \
  --tool-call-parser poolside_v1 \
  --trust-remote-code

TRT-LLM

trtllm-serve poolside/Laguna-S-2.1 --trust-remote-code \
    --tool_parser poolside_v1 --reasoning_parser laguna

Note the flag names differ from vLLM's (--tool_parser, and the reasoning parser is laguna, not poolside_v1).

llama.cpp

GGUF conversions are available at poolside/Laguna-S-2.1-GGUF. Serve with poolside's llama.cpp fork, branch `laguna`, which carries full Laguna support including DFlash speculative decoding. (Base Laguna support is also in upstream review: ggml-org/llama.cpp#25165.)

git clone --branch laguna https://github.com/poolsideai/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000

# with DFlash speculative decoding:
./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \
  -md laguna-s-2.1-DFlash-BF16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 -fa on --jinja --port 8000

Controlling reasoning

Laguna S 2.1 has native reasoning support and works best with preserved thinking: keep reasoning_content from prior assistant messages in the message history. The model will generally reason before calling tools and between tool calls, and may stop reasoning in follow-up steps if prior thinking blocks are dropped.

Thinking is controlled per request via the chat template:

extra_body={"chat_template_kwargs": {"enable_thinking": False}}

or at the server level with --default-chat-template-kwargs '{"enable_thinking": true}'. For agentic coding use cases we recommend enabling thinking and preserving reasoning in the message history.

License

This model is licensed under the OpenMDW-1.1 License.

Intended and Responsible Use

Laguna S 2.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1 is subject to the OpenMDW-1.1 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna S 2.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to security@poolside.ai.