CoolFace
Modelpublic

Yash0707/MathTutor-Qwen3-8B-QLoRA

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes6downloads
Model Card

MathTutor-Qwen3-8B-QLoRA

A QLoRA adapter that turns Qwen3-8B into a K-12 math tutor for algebra, word problems, and arithmetic reasoning.

What it does

  • —Teaches step-by-step with Goal / Key Idea / Steps / Worked Example / Checkpoint structure
  • —Handles student misconceptions, confusion, and frustration
  • —Redirects off-topic questions back to math
  • —41% more concise than base Qwen3-8B while scoring higher on all evaluation metrics

Training

ParameterValue
Base modelQwen3-8B (4-bit QLoRA via Unsloth)
LoRA rank / alpha16 / 16
NEFTune noiseα=5
Dataset612 examples (426 seed-based tutoring + 100 OOD refusals + 86 context-free)
Data sourcesGSM8K (MIT), Orca-Math (MIT), synthetic via Claude Sonnet 4
Epochs2
Batch size16 (effective)
Learning rate2e-4, cosine schedule
Training time~4 min on L40S
Trainable params43.6M (0.83%)

Evaluation (LLM-as-Judge, Claude Sonnet 4)

MetricBase Qwen3-8BMathTutor (this model)Delta
Correctness (in-domain)4.794.88+0.09
Pedagogy (in-domain)4.524.56+0.04
Structure4.804.86+0.06
Faithfulness4.624.80+0.18
Refusal (OOD)3.144.12+0.98

Full evaluation results and methodology: MathTutor-Evaluation

How to use

python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Yash0707/MathTutor-Qwen3-8B-QLoRA",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

messages = [
    {"role": "system", "content": "You are MathTutor, an expert K-12 math tutor specializing in algebra, word problems, and arithmetic reasoning."},
    {"role": "user", "content": "I think 2(x+3) = 2x+3. Is that right?"}
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.6, use_cache=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Links