tcclaviger/Laguna-S-2.1-RFA_L
045
1---2library_name: transformers3inference: false4extra_gated_description: >-5 To learn more about how we process your personal data, please read our <a6 href="https://poolside.ai/legal/privacy">Privacy Policy</a>.7tags:8- laguna-s-2.19- vllm10- quantized11- rfa12- 4-bit13license: openmdw-1.114pipeline_tag: text-generation15base_model:16- poolside/Laguna-S-2.117base_model_relation: quantized18---19 20<h2 align="center">🔥 UPDATED — tokenizer refreshed & quantization improved:<br>layer-0 dense MLP and shared experts now preserved in BF16 <br>Chat Template from: https://huggingface.co/sanjxz/Laguna-S-2.1-Agentic-Chat-Template-Jinja 🔥</h2>21 22---23 24> [!IMPORTANT]25> ## Linear-RFA 4-bit quant of [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1)26>27> This is `tcclaviger/Laguna-S-2.1-RFA_L` — an RFA 4-bit quantization of Poolside's28> **Laguna-S-2.1**. The `_L` (linear-only) build quantizes the MLP and expert linear29> layers to 4-bit (IQ4_NL grid, group 16, Hadamard-16 rotation, asymmetric block-float30> scales) while keeping **all attention, the router gate, the lm_head, the layer-0 dense31> MLP, and the shared experts in bf16**.32> 71.2 GB total, 4.504 bits/weight. All credit for the model to Poolside; this repo only33> changes the numerics. **The full original model card is preserved verbatim below.**34>35> **Runtime:** requires [`tcclaviger/vllm:latest`](https://hub.docker.com/r/tcclaviger/vllm) — an **RDNA 4 (gfx12xx)** vLLM image and the only build with the RFA kernels; no other vLLM build loads these weights. **Not validated on any other hardware at this time.**36>37> **DFlash speculative decoding works with this RFA checkpoint.** Pair it with the38> [Laguna-S-2.1-DFlash](https://huggingface.co/poolside/Laguna-S-2.1-DFlash) draft model39> exactly as documented in the vLLM section below — the RFA quant does not touch the40> attention path, so the DFlash draft head runs unchanged.41>42>**DRY has been incorportaed if desired into the docker image as has repetition blocking guard, two layers of protention. See the help for DRY parameters.43 44 45> [!NOTE]46> ## Evaluation results (this RFA quant)47>48> | Eval | Result |49> |---|---|50> | Throughput | 80.6 tok/s out @ conc 1 (DFlash accept ~2.1); 615.7 tok/s out @ conc 50, ISL 512 |51> | WikiText-2 PPL | 8.109 ± 0.043 (n_ctx 2048, llama.cpp-compatible) |52> | Loglikelihood acc | arc_challenge 0.503 / arc_easy 0.769 / winogrande 0.658 / hellaswag 0.814 (acc_norm) |53> | tool-eval-bench (no-think) | 89/100 (full 69 scenarios) |54> | GSM8K / MMLU / IFEval | 90% / 60% / 95% (prompt-level) |55> | Long-context code recall | 0.980 overall (py 1.00 / js 0.93 / rs 0.99 / cpp 1.00); tool pass 93.5% |56> | Hard Mode agentic (thinking-ON) | 73/100 — 10 pass / 2 partial / 3 fail |57>58> **Test harnesses:** `vllm bench serve` (random dataset, saturation sweep) ·59> llama.cpp-compatible WikiText-2 perplexity · lm-evaluation-harness (loglikelihood,60> local-completions) · tool-eval-bench v2 (69 scenarios + GSM8K/MMLU/IFEval + 1561> Hard Mode scenarios) · codeneedle (positional recall, 4 corpora + tool pass).62>63> **Hardware:** `tcclaviger/vllm:latest`, 4× AMD AI PRO R9700 GPUs (TP4), Ryzen 964> 9950X, 256 GB DDR5-6000. (Not yet fully tuned for throughput.)65 66---67 68<p align="center">69 <img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-s-2-1-banner.svg" width="800px">70</p>71 72<p align="center">73 <a href="https://openrouter.ai/poolside/laguna-s-2.1"><strong>Use on OpenRouter</strong></a> ·74 <a href="https://vercel.com/ai-gateway/models/laguna-s-2.1"><strong>Use on Vercel AI Gateway</strong></a> ·75 <a href="https://poolside.ai/blog/introducing-laguna-s-2-1"><strong>Release blog post</strong></a>76</p>77 78<br>79 80# Laguna S 2.181 82Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated83parameters per token, designed for agentic coding and long-horizon work. It sits84between [Laguna XS 2.1](https://huggingface.co/poolside/Laguna-XS-2.1) (33B-A3B) and85Laguna M.1 (225B-A23B) in the Laguna series and shares the family recipe: a86token-choice router with softplus gating over 256 routed experts plus one shared87expert, grouped-query attention, and interleaved full/sliding-window attention.88 89## Highlights90 91- **Mixed SWA and global attention layout**: 48 layers in a 1:3 global-to-SWA ratio92 (12 global attention layers, 36 sliding-window layers, window 512), with softplus93 attention gating and per-layer-type rotary scales94- **1M context**: 1,048,576-token context window95- **Native reasoning support**: interleaved thinking between tool calls, with96 per-request control via `enable_thinking`97- **Speculative decoding**: a trained98 [DFlash draft model](https://huggingface.co/poolside/Laguna-S-2.1-DFlash) is available99 for lower-latency serving100- **Quantized variants**:101 [FP8](https://huggingface.co/poolside/Laguna-S-2.1-FP8),102 [NVFP4](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4),103 [INT4](https://huggingface.co/poolside/Laguna-S-2.1-INT4) and104 [GGUF](https://huggingface.co/poolside/Laguna-S-2.1-GGUF)105- **OpenMDW-1.1 license**: Use and modify the model and associated materials freely106 for commercial and non-commercial purposes107 ([learn more about OpenMDW](https://openmdw.ai/))108 109## Model overview110 111- Number of parameters: 118B total, ~8B activated per token112- Layers: 48 (12 global attention, 36 sliding-window attention)113- Experts: 256 routed (top-10) plus 1 shared expert114- Attention: grouped-query, 8 KV heads, head dim 128; per-head softplus output gating115- Sliding window: 512 tokens116- Context window: 1,048,576 tokens117- Vocabulary: 100,352 tokens (Laguna family tokenizer)118- Modality: text-to-text119- Reasoning: interleaved thinking with preserved thinking120 121## Benchmark results122 123<p align="center">124 <img alt="benchmarks" src="https://poolside.ai/assets/laguna/laguna-s-2-1-chart.svg" width="800px">125</p>126 127| Model | Size | Terminal-Bench 2.1 | SWE-bench Multilingual | SWE-Bench Pro (Public Dataset) | DeepSWE | SWE Atlas (Codebase QnA) | Toolathlon Verified |128|---|---|---|---|---|---|---|---|129| **Laguna S 2.1** | 118B-A8B | **70.2%** | **78.5%** | **59.4%** | **40.4%** | **46.2%** | **49.7%** |130| Tencent Hy3 | 295B-A21B | 71.7% | 75.8% | 57.9% | - | - | - |131| Inkling | 975B-A41B | 63.8% | - | 54.3% | - | - | 45.5%* |132| Nemotron 3 Ultra | 550B-A55B | 56.4% | 67.7% | - | - | - | 34.3%* |133| DeepSeek-V4-Pro Max | 1.6T-A49B | 64.0%* | 76.2% | 55.4% | 9.0%* | 27.2%* | 55.9%* |134| Kimi K3 | 2800B-A50B | 88.3% | - | - | 69% | - | - |135| Qwen 3.7 Max | - | 74.5%* | 78.3% | 60.6% | - | - | - |136| Muse Spark 1.1 | - | 80% | - | 61.5% | 53.3% | 42.2%* | 75.6% |137| Claude Fable 5 | - | 88% | - | 80.3% | 70% | - | - |138 139Benchmarks as of 21 July 2026. Laguna S 2.1 in **bold**; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI's official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories: [trajectories.poolside.ai](https://trajectories.poolside.ai).140 141## Usage142 143Laguna S 2.1 uses the same `laguna` architecture as Laguna XS 2.1, so the same144engine integrations apply (vLLM, SGLang, Transformers, TRT-LLM, llama.cpp). At 118B145parameters the BF16 checkpoint needs multiple GPUs (roughly 236GB of weights);146quantized variants reduce this substantially.147 148### vLLM149 150```shell151vllm serve \152 --model poolside/Laguna-S-2.1 \153 --tensor-parallel-size 4 \154 --tool-call-parser poolside_v1 \155 --reasoning-parser poolside_v1 \156 --enable-auto-tool-choice \157 --served-model-name laguna \158 --default-chat-template-kwargs '{"enable_thinking": true}'159```160 161> [!NOTE]162> **Optional: speculative decoding with DFlash.** Pair with the163> [Laguna S 2.1 DFlash draft model](https://huggingface.co/poolside/Laguna-S-2.1-DFlash)164> by adding165> `--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":7,"method":"dflash"}'`.166 167### SGLang168 169```shell170python -m sglang.launch_server \171 --model-path poolside/Laguna-S-2.1 \172 --tp-size 4 \173 --reasoning-parser poolside_v1 \174 --tool-call-parser poolside_v1 \175 --trust-remote-code176```177 178### TRT-LLM179 180```shell181trtllm-serve poolside/Laguna-S-2.1 --trust-remote-code \182 --tool_parser poolside_v1 --reasoning_parser laguna183```184 185Note the flag names differ from vLLM's (`--tool_parser`, and the reasoning parser186is `laguna`, not `poolside_v1`).187 188### llama.cpp189 190GGUF conversions are available at191[poolside/Laguna-S-2.1-GGUF](https://huggingface.co/poolside/Laguna-S-2.1-GGUF).192Serve with poolside's llama.cpp fork, branch193[`laguna`](https://github.com/poolsideai/llama.cpp/tree/laguna), which carries194full Laguna support including DFlash speculative decoding. (Base Laguna support195is also in upstream review:196[ggml-org/llama.cpp#25165](https://github.com/ggml-org/llama.cpp/pull/25165).)197 198```shell199git clone --branch laguna https://github.com/poolsideai/llama.cpp200cd llama.cpp && cmake -B build && cmake --build build -j201 202./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000203 204# with DFlash speculative decoding:205./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \206 -md laguna-s-2.1-DFlash-BF16.gguf \207 --spec-type draft-dflash --spec-draft-n-max 7 -fa on --jinja --port 8000208```209 210## Controlling reasoning211 212Laguna S 2.1 has native reasoning support and works best with *preserved thinking*:213keep `reasoning_content` from prior assistant messages in the message history.214The model will generally reason before calling tools and between tool calls, and215may stop reasoning in follow-up steps if prior thinking blocks are dropped.216 217Thinking is controlled per request via the chat template:218 219```python220extra_body={"chat_template_kwargs": {"enable_thinking": False}}221```222 223or at the server level with224`--default-chat-template-kwargs '{"enable_thinking": true}'`. For agentic coding225use cases we recommend enabling thinking and preserving reasoning in the message226history.227 228## License229 230This model is licensed under the [OpenMDW-1.1 License](https://huggingface.co/poolside/Laguna-S-2.1/blob/main/LICENSE.md).231 232## Intended and Responsible Use233 234Laguna S 2.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1 is subject to the [OpenMDW-1.1 License](https://huggingface.co/poolside/Laguna-S-2.1/blob/main/LICENSE.md), and should be used consistently with Poolside's [Acceptable Use Policy](https://poolside.ai/legal/acceptable-use-policy). We advise against circumventing Laguna S 2.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.235 236Please report security vulnerabilities or safety concerns to [security@poolside.ai](mailto:security@poolside.ai).237