Modelpublic

Anupam-Rawat-IITB/T3D-OPDLM-4B-MATH-stage2-checkpoints

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes

Model Card

T3D Stage 2 (DDO) — OPDLM-4B reproduction checkpoints

⚠️ These are INTERMEDIATE checkpoints from an IN-PROGRESS, UNOFFICIAL reproduction. They are not official T3D weights, not an official OPDLM release, and are not validated against any benchmark. Do not treat them as either.

What this is

Per-checkpoint snapshots from an independent reproduction of Stage 2 (DDO training) of the T3D paper (arXiv:2602.12262, code: Tyrion58/T3D).

Deliberate deviation from the paper: the teacher/base model is `divelab/OPDLM-4B` (revision 5e5f00ec75543fa1aa293e22e05d7e9de2c03e0b) instead of the paper's SDAR-4B-Chat. This substitution is the only intentional departure; all other hyperparameters follow the paper/reference code.

Layout

Each folder is a complete, self-contained model (both safetensors shards, index, tokenizer, config) loadable directly:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
# NOTE: OPDLM-4B uses a custom "a2d-qwen3" architecture; see divelab/OPDLM for the
# model class registration required before loading.

—epoch-NN-stepSSS/ — checkpoint at the end of training epoch NN (global step SSS)
—round-NN-stepSSS/ — intermediate snapshot every 10 optimizer steps

Training configuration


Corpus	`OPDLM-4B-MATH_train_stage2` (8,523 trajectories, Gen-Verse/MATH_train)
Epochs	30
Optimizer	AdamW8bit (bitsandbytes), lr 1e-5, cosine, 20 warmup steps
Precision	bf16
Effective batch	256 (1 × 64 grad-accum × 4 GPUs)
block_size	4
DDO	α=0.5, β=0.1, λpath=0.2, multiround (round_interval=10)
Hardware	4 × H100 80GB

Provenance and licensing

Derived from divelab/OPDLM-4B (MIT, base Qwen/Qwen3-4B) and released under the same MIT terms. Credit for the underlying method and base model belongs to the T3D and OPDLM authors respectively; errors in this reproduction are the reproducer's own.