Modelpublic

huluhuluu/qwen3-1p7b-future-hidden-epoch-9-step-233900

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes63downloads

Model Card

Qwen3 1.7B future hidden

This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with Qwen/Qwen3-1.7B. It is a draft model for speculative decoding, not a standalone target language model.

Checkpoint

Field	Value
Source run	`qwen3-1.7b-future-hidden`
Checkpoint	`epoch_9_step_233900`
Epoch	`9`
Global step	`233900`
Files	`config.json`, `model.safetensors`, `training_state.pt` (when present)

Training Parameters

Parameter	Value
Base model	`Qwen/Qwen3-1.7B`
Method	`SpecForge EAGLE3 online training`
Training data	`ShareGPT-format SpecForge training set`
Learning rate	`Not recorded`
Batch size	`Not recorded`
Target batch size	`Not recorded`
Epochs configured	`Not recorded`
Max length	`Not recorded`
Warmup ratio	`Not recorded`
Max grad norm	`Not recorded`
TTT length	`Not recorded`
Draft layers	`Not recorded`
Draft sliding window	`Not recorded`
Save interval	`Not recorded`
Eval interval	`Not recorded`
Seed	`Not recorded`
TP / DP size	`Not recorded / Not recorded`
Attention backend	`Not recorded`
Target model backend	`Not recorded`
SGLang attention backend	`Not recorded`
Dataset build workers	`Not recorded`

Draft Model Configuration

Field	Value
Architecture	`LlamaForCausalLMEagle3`
dtype	`bfloat16`
Hidden size	`2048`
Intermediate size	`6144`
Draft layers	`1`
Attention heads	`16`
KV heads	`8`
Draft vocab size	`32000`
Vocab size	`151936`
Max position embeddings	`40960`
Sliding window	`Not recorded`
Max window layers	`Not recorded`
Future hidden	`True`

Notes

—This is the highest-step local checkpoint available when this repository was published.
—The checkpoint is intended to be loaded by SpecForge/EAGLE3-compatible code.
—training_state.pt is included when available for provenance and training-state inspection.
—No benchmark claim is made in this card.