Modelpublic

meshllm/glm-4.7-flash-spd-sidecar

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes

Model Card

GLM 4.7 Flash SPD Sidecar

This repository contains a trained Speculative Pipeline Decoding (SPD) sidecar head for zai-org/GLM-4.7-Flash, produced for mesh-llm/Skippy PR #866.

The head was trained against a frozen GLM 4.7 Flash checkpoint using the reference SPD trainer from yuyijiong/speculative_pipeline_decoding, with mesh-llm patches for GLM 4.7 topology, reduced-vocab KL training, and usable-row filtering.

Intended Use

This is a research/validation sidecar for verified speculative decoding. The target GLM model remains authoritative and verifies every proposed token. Current Skippy/Rust code can validate the manifest and serving checkpoint shape; live Rust-side SPD execution is still the integration target.

Training Configuration

Field	Value
Base model	`zai-org/GLM-4.7-Flash`
Dataset	`HuggingFaceH4/ultrachat_200k`, split `train_sft`
Raw train rows	`8192`
Usable train rows after shifted-label filtering	`7377`
Stages	`3`
Stage layer boundaries	`15,31,47`
Spec layers	`1`
Draft vocab size	`16000` GLM-tokenizer ids
Max train length	`512`
Epochs	`1`
Optimizer steps	`923`
Batch / grad accumulation	`1 / 8`
Learning rate	`2e-5`, `50` warmup steps
Device	Apple MPS

Verified Acceptance Eval

Evaluation used the donor verifier with target verification enabled, greedy decode (temperature=0.0), draft_top_k=1, and 4 prompts each from MT-Bench, HumanEval, and GSM8K.

Scope	Acceptance	Equivalent accept length	Theoretical gain	Accepted flags	New tokens	Decode steps
Overall	`0.3699`	`1.1098`	`11.09%`	`109/768`	`768`	`2076`
MT-Bench	`0.3596`	`1.0787`	`7.88%`	`26/256`	`256`	`712`
HumanEval	`0.3689`	`1.1066`	`10.76%`	`35/256`	`256`	`694`
GSM8K	`0.3821`	`1.1463`	`14.64%`	`48/256`	`256`	`670`

Files

—train/speculation_head_final.pt: original PyTorch reference checkpoint.
—train/spd-head.safetensors: Rust-readable serving checkpoint exported as F16.
—train/skippy-spd-head.json: Skippy SPD manifest with topology, draft vocabulary ids, and checksums.
—train/trainer_state.json: final trainer state and logged loss/accuracy curve.
—eval/summary/*.json: verifier acceptance summary.
—eval/raw/*.jsonl: per-sample verifier trace.
—data/draft_vocab_top_16000.json: generated GLM-tokenizer draft vocabulary.
—repro/train.sh: training/eval command used for this artifact.
—repro/export.sh: safetensors export command.
—checksums.json: SHA-256 and byte size for uploaded files.

Manifest Validation

The exported manifest was validated with:

bash

SKIPPY_SPD_MANIFEST=/tmp/glm-4.7-flash-spd-sidecar-hub/train/skippy-spd-head.json   cargo test -p skippy-runtime --lib validates_external_manifest_when_skippy_spd_manifest_is_set