Modelpublic

dbtxy/DASH-Qwen3-8B-LoRA

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes10downloads

Model Card

DASH-Qwen3-8B-LoRA

LoRA adapter for Qwen/Qwen3-8B, trained with DASH (Divergence-Adaptive Supervision Horizons), a credit-assignment method for on-policy self-distillation.

DASH replaces the uniform aggregation coefficient of vanilla on-policy self-distillation with adaptive propagation gates: each local distillation signal's gap to the sequence mean drives a backward multi-step recursion, giving every token a path-dependent weight instead of a flat 1/T.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-8B"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "<org>/DASH-Qwen3-8B-LoRA")
tok = AutoTokenizer.from_pretrained("<org>/DASH-Qwen3-8B-LoRA")

msgs = [{"role": "user", "content": "Find the number of ordered pairs..."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=38912,
                     do_sample=True, temperature=1.0, top_p=1.0)
print(tok.decode(out[0], skip_special_tokens=True))

To merge the adapter into the base weights: model = model.merge_and_unload().

Training

Trained on OpenThoughts-Math-30K (29,434 problems with reference solutions). The teacher is the same network with the LoRA adapter disabled, conditioned on the reference solution; the student sees only the problem and rolls out on-policy.


LoRA	r=64, alpha=128, dropout=0.05, on q/k/v/o/gate/up/down projections
Optimizer	AdamW, lr 5e-6, linear decay, no warmup, maxgradnorm 0.1
Global batch	64
Divergence	full-vocabulary forward KL, per-element clip tau=0.05
Gate sharpness	kappa = 5
Rollout	temperature 1.1, topp 0.95, topk 20, max 1024 tokens
Precision	bf16, DeepSpeed ZeRO-2

Code

Training and evaluation code: github.com/DBtxy/DASH-OPSD

Built on siyan-zhao/OPSD.

Citation

bibtex

@inproceedings{TODO_key,
  title     = {DASH: Divergence-Adaptive Supervision Horizons for On-Policy
               Self-Distillation of Reasoning Models},
  author    = {TODO},
  year      = {2027}
}