hbfreed/Laguna-S-2.1-winnow-keep75

Laguna-S-2.1 · Winnow keep-75

poolside/Laguna-S-2.1 (118B-A8B) with 25% of routed-expert channels removed by winnow variable-width expert pruning — 89.2B parameters, no healing/retraining.

Instead of deleting whole experts (REAP-style), winnow shrinks each expert's SwiGLU intermediate width according to a per-channel importance score (router_weight × |post-SwiGLU activation| × down-projection column L2), allocated by a per-layer global rank. 97.7% of experts survive at reduced, ragged widths; the shared expert, attention, and embeddings are untouched.

Scoring, pruning, and evaluation all ran layer-streamed on 3×RTX 3090 (the source model never fit in memory): one decoder layer resident per GPU, residual streams ping-ponged through disk, calibration data-parallel across GPUs. Calibration: 2.1M tokens of bigcode/the-stack-smol.

Results (unhealed)

• —Perplexity: 64 held-out the-stack-smol sequences (2048 tokens each), identical ruler for all three rows, computed layer-streamed.
• —HumanEval: greedy, chat-extract protocol, thinking disabled, max 1280 new tokens, INT8-expert serving. Not comparable to raw-completion HumanEval numbers or to poolside's published agentic benchmarks (SWE-bench, Terminal-Bench 2.1), which use a different harness and far larger budgets.
• —Qualitative agentic check: with a 16k serving context the model autonomously solved several classic Terminal-Bench 1.0 core tasks end-to-end on the same 3×3090 host; most failures were context exhaustion, not task competence.

Serving

At ~95GB INT8 this model does NOT fit on 3×24GB GPUs; it needs ~2×80GB (or 4×40GB) cards, via the winnow vLLM plugin (ragged experts are not representable in stock FusedMoE):

vllm serve hbfreed/Laguna-S-2.1-winnow-keep50 \
  --enforce-eager --pipeline-parallel-size 3 --quantization experts_int8 \
  --max-model-len 16384

Measured on 3×3090: 34 tok/s single-stream, ~150 tok/s at batch 8. Tensor parallelism is unsupported (packed ragged weights don't shard); pipeline parallelism is the intended mode.

Reference (unfused) loading works with plain transformers ≥ 5.13:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "hbfreed/Laguna-S-2.1-winnow-keep50", trust_remote_code=True)

winnow.json records the full pruning plan, calibration provenance, and per-expert channel indices.

Pruned with winnow's stream-prune; see the winnow repository for the method writeup. Base model by poolside, released under OpenMDW-1.1; this derivative keeps that license.