albertge/llada-8b-full-sft-mix60k-code-codetags

LLaDA-8B full-sequence SFT with code-target continuation

Matched full-sequence-SFT code control for the dLLM Registers paper. This checkpoint has no register or discrete-text carry channel.

Training

• —Initialization: albertge/llada-8b-full-sft-mix60k-4pass
• —Continuation data: the 30K OpenCodeInstruct subset of albertge/mix60k-math-code-sft
• —Targets: raw Python delimited by <code>...</code>
• —Four independently noised diffusion-loss optimizer passes per example
• —Natural/dynamic sequence lengths, capped at 1024 completion tokens
• —One epoch, batch size 1 per rank, learning rate 1e-5, weight decay 0.1
• —Exact runner: SFT/sft_train_vanilla.py and modal_jobs/vanilla_sft.py

Recorded bounded evaluation

Fresh 16 x 64 clears generated text at every boundary and carries no state. Chunks are concatenated and scored as one program.

Repository: https://github.com/lbertge/d1-registers