CoolFace
Modelpublic

JANGQ-AI/Laguna-S-2.1-JANG_6M

sourceHugging Faceotherupdated 19d agoView on Hugging Face
1likes102downloads
Model Card

<div align="center"> <img src="jangq-logo.png" height="80" alt="JANGQ-AI"/> </div>

<p align="center"> <a href="https://vmlx.net"><img src="vmlx-app.png" alt="vMLX — run JANG models on Apple Silicon" width="820"></a> </p>

<h3 align="center">⚡ All JANG models are meant to be run in <a href="https://vmlx.net">vMLX</a></h3>

JANGQ-AI/Laguna-S-2.1-JANG_6M

JANG_6M JANG affine bundle of poolside/Laguna-S-2.1 — 118B-parameter / ~8B-active MoE for agentic coding and long-horizon work, quantized for Apple Silicon.

Attention (incl. the softplus output gate), shared expert, router, and norms stay high precision; the 256-routed-expert bulk carries 6-bit gate/up/down, protected by activation-aware (AWQ) scales chosen by measured quantization error against a no-AWQ baseline (3.1% lower reconstruction error at this bit width).

Bundle

FieldValue
Sourcepoolside/Laguna-S-2.1 @ e80da38
Architecturelaguna — 48 layers (12 global + 36 SWA w512), 118B-A8B, 256 experts top-10 + shared, 1M ctx
On-disk size96.5 GB (21 shards)
Routed experts6-bit gate/up/down affine, group 64, AWQ folded
Attention q/k/v/o + g_proj8-bit affine
Shared expert / dense FFN8-bit affine
Embeddings / lm_head6-bit / 8-bit affine
Router, e_score bias, normsfp16 passthrough
Modalitytext-only (verified from tensor index)

Measured (M5 Max, 128 GB, greedy, wired)

MetricValue
Decode30.7 tok/s
Long-context cache parityteacher-forced top-1 agreement 1.000 / 1.000 (pre/post the 512 sliding window, 2,913-token pass)

Chat / reasoning

  • —GLM-style think tags; enable_thinking toggles reasoning (thinking is ON by default in this revision's template; pass enable_thinking=False to disable)
  • —The FULL chat template is inlined in tokenizer_config.json (upstream ships only an {% include %} stub that most runtimes cannot resolve — inlining is what makes the reasoning toggle actually work)
  • —Stop tokens eos_token_id = [2, 24] — id 24 is end-of-turn and must be in the stop set
  • —Template emits its own leading 〈|EOS|〉 (bos 2): do not prepend another
  • —Tool calls: <tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>
  • —Vendor sampling: temperature 1.0, topp 1.0, topk 20

Loaders must honor the per-module {bits, group_size, mode} overrides in config.json[quantization].

Quantized and verified by Jinho Jang (eric@jangq.ai). License: OpenMDW-1.1 (inherited).