manjunathshiva/Laguna-XS.2-tq3-g64

Laguna-XS.2 — TurboQuant 3-bit (MLX)

Data-free 3-bit quantization of poolside/Laguna-XS.2 for Apple Silicon via TurboQuant-MLX. Laguna XS.2 is a 33B-parameter Mixture-of-Experts (256 experts, top-8, 3B active) built by Poolside for agentic coding and long-horizon work on a local machine.

• —13.8 GB on disk, ~16 GB peak — runs comfortably on a 32 GB Mac (Poolside's own card quotes 36 GB for the bf16). Under Metal's ~75–81 % working-set limit, the practical floor is 32 GB with headroom.
• —Data-free: no calibration set. Randomized Hadamard rotation → per-group RMS scale → Lloyd-Max codebook (optimal for the rotated Gaussian), decoded inline by fused Metal kernels.
• —Agentic-capable: passes the Opencode observe → read → edit → verify loop.

How to run

pip install "turboquant-mlx-full>=0.18.1"  # Laguna support; 0.17.0/0.18.0 are yanked
# generate
python -m turboquant_mlx.generate --model <this-repo> \
    --prompt "Write a mergesort in Python." --temp 0.7 --top-p 0.9
# OpenAI-compatible server (agentic use)
turboquant-serve --model <this-repo> --port 8080 \
    --temp 0.7 --top-p 0.9 --prompt-concurrency 1 \
    --chat-template-args '{"enable_thinking": false}'

Always pass `--top-p 0.9`. Laguna's generation_config ships top_p: 1.0; the untruncated 100k-vocab tail injects rare junk tokens even in otherwise clean output. Do not pass `--kv-bits`. 30 of 40 layers are sliding-window(512), so KV stays tiny (peak 16.8 GB @2K → 19.4 GB @31K). KV-quant saves nothing here and can cost up to 4.4× decode. The transformers tokenizer warning is harmless — ignore it. Recent transformers prints "incorrect regex pattern … this will lead to incorrect tokenization" and suggests fix_mistral_regex=True. Don't. This repo's tokenizer.json is structurally identical to the upstream Poolside release (same vocab, merges and pre-tokenizer), so the default is how the model was trained and what every other runtime uses. Setting the flag changes case-aware splitting (apoA, GeV, iPhone) on ~1% of real prompts and would make results diverge from every other Laguna deployment. All benchmarks below were run without it.

<!-- turboquant-plan:begin -->

Will it fit my Mac?

Don't guess, and don't download 14.9 GB to find out — ask first (needs turboquant-mlx-full >= 0.18.1):

turboquant-plan --model manjunathshiva/Laguna-XS.2-tq3-g64

It reads only this repo's safetensors headers over the network (a couple of hundred KB, a few seconds — the weights are never fetched), then projects the peak against your machine and prints the flags to use. On a 64 GB M4 Max:

Model
  type                 laguna  (turboquant 3-bit g64)
  weights (exact)      14.85 GB

Machine
  Metal working set    55.66 GB   ← the real ceiling
  system RAM           68.72 GB

Projection at 16,384 tokens of context
  weights              14.85 GB
  KV cache             0.73 GB  (43.8 KB/token, hybrid: 10/40 full-attention, 30 sliding (window 512))
  prefill workspace    3.22 GB  (estimate, at --prefill-step-size 2048)
  runtime reserve      1.00 GB  (buffer cache, activations, fragmentation)
                       ----------------------------------
  peak                 19.81 GB of 61.85 GB usable   42.04 GB headroom

Verdict: ✅ RESIDENT — fits fully in memory

Planning for a machine you're not sitting at — deciding whether a 32 GB Mac can take this — is what --wired-gb / --ram-gb are for:

turboquant-plan --model manjunathshiva/Laguna-XS.2-tq3-g64 \
    --wired-gb 24 --ram-gb 32 --context 8192

Of the 14.85 GB, 13.62 GB is streamable experts and only 1.24 GB has to stay resident, so on a machine too small to hold it the planner recommends --streaming rather than giving up. (Laguna's expert container is recognised as streamable from turboquant-mlx-full >= 0.18.1; earlier releases counted it as resident and reported ❌ on small Macs.)

turboquant-doctor runs the same projection plus a readiness check; both take --json. The projection is calibrated against real measurements on a 16 GB Mac mini rather than estimated from theory — on the 9.4 GB ternary 35B it predicts a 10.44 GB peak where that machine measures 10.42.

<!-- turboquant-plan:end -->

Quality (6-prompt stress battery, temp 0.7 / top-p 0.9)

Standard accuracy (resident, no-think, temp 0.7; harness in scripts/benchmarks/):

Agentic coding (Opencode)

Passes the full agentic loop on the planted-bug fix task (run tests → locate bug → minimal edit → re-run → confirm). Serving notes:

• —Serve with --top-p 0.9, --prompt-concurrency 1, enable_thinking:false.
• —Tool calls use Laguna's native GLM-XML format; TurboQuant-MLX ships the parser fix for the trailing-newline function-name bug in mlx-lm's glm47 parser.

Speed & the 3-bit vs 4-bit tradeoff

Measured on an M4 Max (64 GB), resident, greedy. TurboQuant's data-free codebook 3-bit is smaller than MLX-native affine 4-bit but ~2× slower per token — because it trades bytes for compute: every forward pass decodes a Lloyd-Max codebook and applies an online Hadamard rotation, work that affine 4-bit (mx.quantized_matmul) does not. This is not a bandwidth effect — 3-bit reads fewer bytes; the cost is the extra per-token FLOPs, which is why the ~2× gap is uniform across prefill and decode:

Both pass Opencode with identical correctness. Pick this 3-bit build for the smallest download and lowest memory (32 GB Macs); pick [affine 4-bit](https://huggingface.co/mlx-community/Laguna-XS.2-4bit) when you want the fastest interactive / agentic loop.

Peak memory vs context (this build):

16× the context costs only +2.6 GB — the sliding-window layout keeps KV nearly flat.

Provenance

• —Base: poolside/Laguna-XS.2, released by Poolside under Apache-2.0; a verbatim copy of that licence ships here as LICENSE.md and applies to these derived weights. (Note for anyone comparing repos: Poolside's later Laguna S-2.1 is not Apache-2.0 — it is released under OpenMDW-1.1. The licence differs per model, so check per repo.)
• —Converter: python -m turboquant_mlx.convert --hf-path poolside/Laguna-XS.2 --mlx-path out --bits 3 --group-size 64 --streaming.
• —MLX Laguna support (model port + loader) is provided by TurboQuant-MLX's compat shim; mlx-lm has no native laguna model class as of 0.31.3.

Copyright 2026 Manjunath Janardhan. Model weights derive from poolside/Laguna-XS.2 and remain subject to the Apache-2.0 licence in LICENSE.md; Poolside's copyright and attribution notices are retained per that licence.