DaoCloud/Ornith-1.5-35B-A3B-DFlash2-2.6B-A0.3B-NVFP4

Ornith-1.5-35B-A3B-DFlash2-2.6B-A0.3B-NVFP4

An NVFP4 Mixture-of-Experts DFlash2 drafter for `ornith-ai/Ornith-1.5-35B-A3B` with a trained confidence head. The routed experts are quantized to NVFP4; every other module stays BF16.

Results

All results below use this NVFP4 drafter. Each benchmark is measured twice: once against the BF16 target `ornith-ai/Ornith-1.5-35B-A3B`, and once against the NVFP4 target `ornith-ai/Ornith-1.5-35B-A3B-NVFP4`.

Accepted length is calculated from the raw serving counters, p_i is the fraction of draft rounds that survive through draft position i:

p_i = accepted_per_pos[i] / draft_rounds
accepted_length = 1 + sum_i p_i

Quantization

Architecture

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Ornith-1.5-35B-A3B or a compatible quantized version:

vllm serve ornith-ai/Ornith-1.5-35B-A3B-NVFP4 \
  --tensor-parallel-size 1 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/Ornith-1.5-35B-A3B-DFlash2-2.6B-A0.3B-NVFP4",
    "num_speculative_tokens": 7
  }'

The checkpoint currently requires a vLLM build containing:

• —`vllm-project/vllm#54154` — sample_from_anchor=true query layouts for DFlash draft models.
• —`vllm-project/vllm#54475` — MoE DFlash2 draft models.
• —`vllm-project/vllm#55668` — optional DFlash2 confidence-head loading.

Checkpoint

The checkpoint contains 6,971 tensors and 2,638,552,065 parameters.

Acknowledgements

This work builds on `ornith-ai/Ornith-1.5-35B-A3B`, vllm-project/speculators, vllm-project/vllm, and NVIDIA/TensorRT-Model-Optimizer.

License

Apache-2.0.