Modelpublic

nerkyor/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-GGUF

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

3likes295downloads

Model Card

Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-GGUF

Llama.cpp MUST READ BEFORE LAUNCH

Official llama.cpp prebuilt binaries without this compatibility patch will fail to load this GGUF.

Reason: unpatched official llama.cpp builds cannot recognize the pruned Step35 / Step 3.7 MoE layout where the routed expert count differs by layer. They still read step35.expert_count as a single u32, while this pruned MoE stores expert counts as a per-layer array.

For example, llama-b9892-bin-win-vulkan-x64 fails during hyperparameter loading with:

text

key step35.expert_count has wrong type arr but expected type u32

This is not a corrupted-shard issue and is not caused by the MTP sidecar. Use a patched Step 3.7 / LynnStyle-compatible build, or apply the patch and rebuild.

If you are not comfortable rebuilding llama.cpp, give the Patch and Usage notes links below to _Codex or another coding agent_, and ask it to apply the patch to your target llama.cpp source tree and rebuild.

Required file	Repository link
Patch	patches/llama.cpp-step37-expert-count-array.patch
Usage notes	runtime/llama.cpp-step37-compat.md

Reference runtime: llama.cpp 8c146a8366304c871efc26057cc90370ccf58dad; the release tests used CUDA llama-server built from the patched llama.cpp tree.

For newer official source trees or binaries such as b9892, apply the same patch logic and rebuild; re-downloading GGUF shards will not fix this error.

Important GGUF parser note: Hugging Face / ModelScope GGUF auto-detection may parse a standalone MTP sidecar or one shard and show misleading metadata such as 3B params, 5-bit, or 2.49GB. That is not the main model. This repository uses multi-shard mixed-precision LynnStyle GGUF. Download the complete Q4/, Q5/, Q8/, or Q3/ shard set plus the matching MTP sidecar, and follow the README launch template. The platform hardware-compatibility panel is not authoritative for this mixed-precision release.

[image]

_Naming note: `GLM5.2-SFT / GPT5.5-RL` describes post-training data provenance, not weight provenance. Most SFT Agent/ReAct trajectories were generated, rewritten, or cleaned with GLM 5.2 API assistance; most RL preference pairs and judge signals came from a GPT-5.5/Codex-style review policy. No GLM or GPT weights are mixed into this model._

Repository Scope

This is the GGUF repository for Step-3.7-Flash-180B-LynnStyle-GLM5.2-SFT-GPT5.5-RL.

The 180B label comes from the pruned effective MoE scale: after expert-workload analysis and task-scope protection, about 8%, roughly 1000 low-contribution experts, were removed from the original MoE.

LynnStyle is the full local-inference method chain: localization-guided protection of key layers, experts, and tensor families; dynamic layered pruning of low-contribution paths; targeted SFT/RL to reduce overthinking, empty answers, non-delivery, and parse failures; and calibrated mixed-precision quantization that keeps core layers, hot experts, router paths, attention, lm_head, and MTP protected.

This repository is for GGUF quantized files, runtime smoke results, MTP single/concurrent checks, and quantized gate metrics. BF16 weights belong in the main repository.

Text Scope and MTP Sidecars

This release currently claims text reasoning, code, and Agent/ReAct text capability. It does not claim image input. The original dyn263 BF16 source retained multimodal traces such as vision_config and vision_encoder.py, but this final RL2 merge, export, quantization, and evaluation path did not validate the complete vision stack: vision-tower weights, projector / mmproj, image processor, image-token alignment, and llama.cpp image-input smoke tests.

MTP sidecars are optional speculative-decoding draft files. They accelerate inference together with a main GGUF model; they are not vision files and cannot be used as standalone main models.

—LynnStyle Q8 uses Q8/Step-3.7-Flash-MTP-Q8_0.gguf.
—LynnStyle Q5 uses Q5/Step-3.7-Flash-MTP-Q8_0.gguf.
—LynnStyle Q4 uses Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf.

LynnStyle GGUF Strategy

Q8 is the quality baseline. The most important endpoint tier in this release is LynnStyle Q4: around 90GB, aimed at fitting a 180B-class MoE into a 96GB/R6000 single-card setup.

The previous way to fit this model class into R6000 was closer to a quality-collapsed Q3. LynnStyle Q4 is tested early because it decides whether the 96GB endpoint route is real.

LynnStyle Q4 Profile

LynnStyle Q4 Profile F6 is not a uniform IQ4_XS file. It is explicit core protection plus interleaved imatrix calibration plus lower precision for cold MoE experts. The dry-run target is 93216.74 MiB / 91.03 GiB, leaving a more realistic margin for a 96GB/R6000 setup.

Protection Bucket	Tensor Family	Precision	Matched Tensors
`L0-L2` first 3 Dense blocks	all `blk.0-2.*.weight`	`Q8_0`	36
Embedding / output / rope / output_norm anchors	`token_embd.weight`, `output.weight`, `output_norm.weight`, `rope_freqs.weight`	`Q8_0`	4
Norm / gate / router small tensors	`attn_norm`, `ffn_norm`, `attn_q_norm`, `attn_k_norm`, `attn_gate`, `ffn_gate_inp`, `exp_probs_b.bias`	`Q8_0`	294
Shared experts	`ffn_down_shexp.weight`, `ffn_gate_shexp.weight`, `ffn_up_shexp.weight`	`Q8_0`	126
Late `L43-L44` MoE experts	`ffn_down_exps.weight`, `ffn_gate_exps.weight`, `ffn_up_exps.weight`	`Q5_K`	6
`L39-L42` shoulder and `L4/L16/L30` hot-zone MoE experts	`ffn_down_exps.weight`, `ffn_gate_exps.weight`, `ffn_up_exps.weight`	`Q4_K`	21
Attention `q/k/v/o` family	`attn_q.weight`, `attn_k.weight`, `attn_v.weight`, `attn_output.weight`	`Q4_K`	168
Cold MoE experts `L5/L6/L8-L11`	`ffn_down_exps.weight`, `ffn_gate_exps.weight`, `ffn_up_exps.weight`	`Q3_K`	18
Remaining MoE experts	other `ffn_down/gate/up_exps.weight`	default `IQ4_XS`	81

The protection table is combined with a 2026-07-06 interleaved imatrix calibration set that rotates through LBC, Coding100, ReAct, GPQA, MMLU, native SFT, and RL-preference prompts. It does not use benchmark answer keys as supervised training labels.

[image]

These figures are the evidence behind the pruning and quantization policy: MMLU/GPQA and LBC/Coding100 do not activate exactly the same experts; L39-L44 are high-risk in both scopes; code tasks expose specialized experts that reasoning-only traces do not fully cover. LynnStyle is therefore not blind pruning and not uniform low-bit quantization.

Available Tiers

Tier	Target Hardware	Role	Status
LynnStyle Q8	190GB+ / multi-GPU	quality baseline	accepted
LynnStyle Q4	96GB / R6000	core recommendation	accepted Profile F6, imatrix, MTP matrix complete
LynnStyle Q5	128GB	high-quality tier	accepted, MTP matrix complete
LynnStyle Q3	64G	lower-memory tier	with Q4 MTP
LynnStyle Q2 Experimental	48GB	experimental entry tier	not published

Download guidance:

—LynnStyle Q4: download every shard under Q4/, from 00001-of-00005 to 00005-of-00005.
—LynnStyle Q5: download every shard under Q5/, from 00001-of-00005 to 00005-of-00005.
—LynnStyle Q8: download every Q8 shard under Q8/.
—MTP sidecar: for Q8, download Q8/Step-3.7-Flash-MTP-Q8_0.gguf; for LynnStyle Q5, download Q5/Step-3.7-Flash-MTP-Q8_0.gguf; for LynnStyle Q4, download Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf.

Metrics

Final RL2 Q8+MTP:

Metric	Result
MMLU500	465/500 = 93.0%
GPQA198	149/198 = 75.25%
Coding100	76/100
LBC100	77/100

LynnStyle Q4 imatrix Profile F6:

Metric	Result
MMLU500	463/500 = 92.6%
GPQA198	142/198 = 71.72%
Coding100	76/100
LBC100	73/100

LynnStyle Q5 imatrix:

Metric	LynnStyle Q5 Result
MMLU500	460/500 = 92.0%
GPQA198	143/198 = 72.22%
Coding100	77/100
LBC100	74/100

[image]

RL2 is the final release because it preserves the main public gates while reducing heavy-thinking and non-delivery failure modes. An RL1 comparison run was stopped at 81/117 = 69.23% on GPQA, making RL2 the stronger release for this objective.

MTP Recommendation

LynnStyle Q4 has completed the C1/C2/C4 by draft N=1..4 MTP matrix on the 2026-07-06 R6000 setup. TPS is llama.cpp predicted_tokens_seconds.

Concurrency	draft N=1	draft N=2	draft N=3	draft N=4	Recommendation
C1	4/4, 124.0 tok/s, 4.816s	4/4, 135.7 tok/s, 4.249s	4/4, 142.3 tok/s, 4.041s	4/4, 141.4 tok/s, 4.066s	N=3
C2	4/4, 76.7 tok/s, 8.015s	4/4, 79.0 tok/s, 7.733s	4/4, 86.4 tok/s, 6.533s	4/4, 84.5 tok/s, 6.417s	N=3
C4	4/4, 56.2 tok/s, 10.164s	4/4, 54.6 tok/s, 10.159s	4/4, 51.1 tok/s, 10.115s	4/4, 51.6 tok/s, 11.220s	N=3

For 96GB/R6000 LynnStyle Q4, use Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf as the draft model. LynnStyle Q5 and Q8 use the Step-3.7-Flash-MTP-Q8_0.gguf sidecar in their own tier directories. The Q8_0 MTP sidecar is about 3.71GB and is not the default Q4 pairing.

LynnStyle Q5 MTP:

Concurrency	Recommended draft N	TPS	Avg latency
C1	N=3	124.1 tok/s	4.372s
C2	N=3	72.0 tok/s	7.579s
C4	N=1	52.2 tok/s	10.976s

File-size fit notes:

—LynnStyle Q4: about 93.16 GiB including its Q5KM MTP sidecar; recommended for 96GB/R6000 with conservative context first.
—LynnStyle Q5: about 123.23 GiB including its Q8_0 MTP sidecar; recommended for 128GB-class memory.
—LynnStyle Q8: about 182.55 GiB including its Q8_0 MTP sidecar; use multi-GPU / 190GB+ class memory.

llama.cpp Templates

LynnStyle Q8 + MTP

bash

MAIN_GGUF="Q8/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q8_0-00001-of-00005.gguf"
MTP_GGUF="Q8/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 32768 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q5 imatrix + MTP

bash

MAIN_GGUF="Q5/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q5-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q5/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q5 imatrix + MTP

bash

MAIN_GGUF="Q5/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q5-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q5/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q4 imatrix + MTP

bash

MAIN_GGUF="Q4/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q4-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000