LLM-OS-Models2/Qwen3-0.6B-diffusion-mdlm-justgrpo-run5-gsm8k-checkpoints

Run 5 - MDLM GSM8K single-domain JustGRPO, ALL checkpoints (10-step interval)

LoRA adapters saved every 10 steps (ckpt-000010 ... ckpt-000200) so you can pick the best step yourself. Key finding: the model OVERTRAINS - ckpt-100 is the peak, the final ckpt-200 is worse. Load an adapter onto the base with PEFT.

Base: dllm-hub/Qwen3-0.6B-diffusion-mdlm-v0.1 (GSM8K base = 29.3)

Measured performance (more filled in as GPU frees)

Other checkpoints/tasks: eval in progress (see the GitHub repo BENCHMARKS.md).

Usage

import torch
from transformers import AutoModelForMaskedLM
from peft import PeftModel
base = "dllm-hub/Qwen3-0.6B-diffusion-mdlm-v0.1"
m = AutoModelForMaskedLM.from_pretrained(base, trust_remote_code=True, torch_dtype=torch.bfloat16)
m = PeftModel.from_pretrained(m, "LLM-OS-Models2/Qwen3-0.6B-diffusion-mdlm-justgrpo-run5-gsm8k-checkpoints", subfolder="ckpt-000100")  # pick your step

Generate with the dllm MDLM sampler (block_size=256). Code: https://github.com/LLM-OS-Models/JustGRPO