CoolFace
Modelpublic

MikhailRudenko/drafter-mixed-ut

sourceHugging Faceapache-2.0updated 27d agoView on Hugging Face
0likes30downloads
Model Card

drafter-mixed-ut

Domain-specific draft model for speculative decoding, trained on Mixed (Understanding + Text Reformulation) tasks.

Model Details

ParameterValue
Base modelLite-Mistral-150M-v2-Instruct (156M params)
ArchitectureMistralForCausalLM
Target modelTurboSparse-Mistral-Instruct (7B)
DomainMixed (Understanding + Text Reformulation) (32 Flan clusters)
Training samples722K
Epochs5
Training time12.6 hours (1x RTX 3090)
LossMixed: 0.5 x CE + 0.5 x KL (T=1.0)
Final eval_loss1.788
Final top1_accuracy56.64%
Overlap Area (AR proxy)0.7191 on own domain

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("MikhailRudenko/drafter-mixed-ut")
tokenizer = AutoTokenizer.from_pretrained("MikhailRudenko/drafter-mixed-ut")

Training Data

MikhailRudenko/domain-aware-sd-synthetic

Citation

Part of the Domain-Aware Speculative Decoding research project: GitHub