CoolFace
Modelpublic

RedHatAI/GLM-5.2-speculator.dspark

sourceHugging Faceapache-2.0updated 26d agoView on Hugging Face
9likes1.8kdownloads
Model Card

RedHatAI/GLM-5.2-speculator.dspark

This is a preliminary (and subject to change) DSpark speculator model for RedHatAI/GLM-5.2-NVFP4-FP8.

It was trained using the Speculators library on the mgoin/open-perfectblend-glm5.2-regen dataset.

Note:

It was validated on Nvidia B200, other hardware validation pending.

Training Details

Launch training

bash
scripts/train.py \
  --verifier-name-or-path RedHatAI/GLM-5.2-NVFP4-FP8 \
  --data-path ./output/dspark_glm52 \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./output/dspark_glm52/checkpoints \
  --epochs 1 \
  --lr 3e-4 \
  --total-seq-len 8192 \
  --speculator-type dspark \
  --block-size 8 \
  --max-anchors 1024 \
  --num-layers 3 \
  --target-layer-ids 2 20 39 58 75 \
  --markov-rank 256 \
  --markov-head-type vanilla \
  --enable-confidence-head \
  --confidence-head-with-markov \
  --loss-fn '{"ce": 0.1, "tv": 0.9}' \
  --confidence-head-alpha 1.0 \
  --fsdp-shard \
  --logger tensorboard \
  --log-dir ./output/dspark_glm52/logs \
  --on-missing generate \
  --on-generate delete \
  --checkpoint-freq 0.01 \
  --request-timeout 60 \
  --max-retries 2

Deployment

Deploy with vLLM using the speculator as a draft model.

bash
vllm serve RedHatAI/GLM-5.2-NVFP4-FP8  \
  --max-model-len 20480 \
  --spec-model RedHatAI/GLM-5.2-speculator.dspark \
  --spec-method dspark \
  --spec-tokens 7 \
  -tp 8

Preliminary Evaluations

Per-Position Acceptance Rate

DatasetPos 0Pos 1Pos 2Pos 3Pos 4Pos 5Pos 6Avg. Length
HumanEval77.8%59.1%44.6%33.9%26.1%20.3%15.8%3.78
math_reasoning87.5%76.6%67.6%59.2%51.8%45.1%39.5%5.27
qa70.7%48.5%33.7%23.7%17.1%11.9%8.4%3.14
question73.7%52.6%38.1%28.5%21.6%16.7%13.2%3.44
rag78.5%61.3%48.7%38.9%30.7%24.2%19.2%4.01
summarization77.9%58.0%43.0%31.7%22.8%16.2%11.4%3.61
tool_call71.9%51.1%37.1%27.6%21.4%16.8%13.2%3.39
translation76.8%57.4%43.5%33.6%26.1%20.0%15.1%3.72
writing74.0%52.2%38.0%28.1%21.3%16.7%13.1%3.44
RedHatAI/GLM-5.2-speculator.dspark · CoolFace