CoolFace
Modelpublic

abenzerps/ZDTaichu5.0-9B-DSpark-GGUF

sourceHugging Faceapache-2.0updated 11d agoView on Hugging Face
1likes1.2kdownloads
Model Card

ZDTaichu5.0-9B DSpark GGUF

GGUF files for ZDTaichu5.0-9B-DSpark, the official semi-autoregressive speculative decoding draft model developed by the Zi Dong Tai Chu team to accelerate ZDTaichu5.0-9B.

[!NOTE] This is a speculative decoding draft model (D-Spark / DFlash), not a standalone general model. It must be paired with the primary ZDTaichu5.0-9B GGUF model during inference to increase generation speeds by up to 60–85%.

Benchmarks

[image]

Benchmark results reported by TaichuAI for the original ZDTaichu5.0-9B foundation model. These figures are not measurements of this GGUF conversion.

GGUF files

**Quantization****DSpark Draft File****Draft Size****Paired Base Model File**
BF16**ZDTaichu5.0-9B-DSpark-BF16.gguf**4.40 GBTaichuAI/ZDTaichu5.0-9B (Original Weights)
Q8_0**ZDTaichu5.0-9B-DSpark-Q8_0.gguf**2.34 GBZDTaichu5.0-9B-Q8_0.gguf (9.53 GB)
Q6_K**ZDTaichu5.0-9B-DSpark-Q6_K.gguf**1.81 GBZDTaichu5.0-9B-Q6_K.gguf (7.36 GB)
Q5_K_M**ZDTaichu5.0-9B-DSpark-Q5_K_M.gguf**1.67 GBZDTaichu5.0-9B-Q5_K_M.gguf (6.47 GB)
Q4_K_M**ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf**1.53 GBZDTaichu5.0-9B-Q4_K_M.gguf (5.63 GB)
Q4_0**ZDTaichu5.0-9B-DSpark-Q4_0.gguf**1.51 GBZDTaichu5.0-9B-Q4_0.gguf (5.31 GB)

Draft model architecture

  • —Framework: DSpark (semi-autoregressive DFlash backbone + chained Markov head).
  • —Target Layers: Extracts auxiliary hidden states from target layers [1, 8, 15, 22, 29].
  • —Block Size: 8 tokens drafted per forward pass (dflash.block_size = 8).
  • —Target Model: ZDTaichu5.0-9B GGUF.

Usage

Use this draft model alongside the primary target model abenzerps/ZDTaichu5.0-9B-GGUF with llama.cpp (llama-cli or llama-server):

Text generation with Speculative Decoding (llama-cli)

bash
llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  -c 8192 --jinja \
  --temp 0.7 --top-p 0.95 \
  -p "Explain why reproducible builds matter."

Multimodal vision understanding with Speculative Decoding (llama-cli)

bash
llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  --image path/to/image.jpg \
  -p "Describe what is shown in this image in detail."

OpenAI-compatible server with Speculative Decoding (llama-server)

bash
llama-server \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  -md ZDTaichu5.0-9B-DSpark-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 8 \
  -c 8192 --jinja -fa on --host 0.0.0.0 --port 8080

Increase context -c up to 131,072 tokens (128K) when sufficient memory is available.

Source and build