trillionlabs/Trida2.0-4B

trillionlabs/Trida2.0-4B

English | [한국어](./README_ko.md)

Trida2.0-4B is a two-stream block-diffusion language model (research checkpoint). It is produced by continued SFT with the full-mask recipe — 100% of each block masked (single view, no complementary mask) at block size bd=8, fresh optimizer, LR 5e-5, ~2k steps — starting from the base SFT checkpoint (v6, step_18000).

Code

Training and serving code lives in [trillion-labs/trida-stack](https://github.com/trillion-labs/trida-stack) — the two-stream block-diffusion training stack (train/) and the HybridDiffusion SGLang serving fork (inference/). See the getting-started notebooks there for first-run training and inference.

How to serve

Best served self-speculative (draft with the diffusion stream, verify with the clean/AR stream): verify-guaranteed AR quality, and — because full-mask training shapes exactly the single-shot full-block prediction a self-spec draft needs — a higher acceptance length than the base, so it's the fastest self-spec at low concurrency. It also serves in plain AR/causal mode. Pure iterative diffusion decoding is not supported by this recipe (the shift decoder's partial-mask states are untrained). Runs on the HybridDiffusion SGLang fork. mask_id = 248077, vocab 248078.

Accuracy

Concurrency 1, threshold 0.9, reasoning on. A vanilla Qwen3.5-4B baseline is being added.

Serving speed & parallel commit (concurrency 1)

Block-diffusion self-spec commits multiple tokens per forward pass at verify-guaranteed AR quality. Trida2.0-4B's full-mask training makes it a better drafter, so it accepts more per step and runs faster than the base.

Per-benchmark commit rate (Trida2.0 self-spec, tokens/forward): GSM8K 3.10, BFCL 2.33, MMLU-Pro 2.33.