rswaminathan38/llmbench-student-3b-gsm8k-full-kd-ft-teacher-20260410

libraryname: transformers pipelinetag: text-generation base_model: meta-llama/Llama-3.2-3B datasets:

• —gsm8k tags:
• —gsm8k
• —transformers
• —vllm
• —text-generation
• —student-model
• —knowledge-distillation ---

Student 3B Full KD

This repo contains the full-logit KD export for the student model from the GSM8K workflow in this project.

Quick Use

Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rswaminathan38/llmbench-student-3b-gsm8k-full-kd-ft-teacher-20260410"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")

vLLM:

vllm serve rswaminathan38/llmbench-student-3b-gsm8k-full-kd-ft-teacher-20260410 --dtype auto

Metrics

• —Test relaxed exact-match accuracy: 0.2494
• —Correct / examples: 329 / 1319
• —Avg generated tokens: 247.7437
• —Prompt style used during evaluation: cot_step_by_step

Training Details

• —Base model: meta-llama/Llama-3.2-3B
• —Variant: full-logit KD
• —Output source: /storage/ice1/3/3/rswaminathan38/LLM_Bench/LLMOptimization/Model_Optimizations/outputs/hf_full_kd_teacher_2026-04-10
• —Teacher used for distillation: Fine-tuned GSM8K teacher 8B (2026-04-10)
• —num_train_epochs: 10
• —per_device_train_batch_size: 2
• —gradient_accumulation_steps: 8
• —learning_rate: 1e-05
• —warmup_ratio: 0.05
• —max_seq_length: 1024
• —alpha: 0.7
• —temperature: 2.0
• —kd_method: full
• —top_k: 64

Notes

• —Prompt format in this repo is question + "\n\nLet's think step by step.\n".
• —Original Meta Llama license and access requirements still apply to downstream use.
• —run_config.json and eval/test_summary.json are uploaded alongside the weights when available.