CoolFace
Modelpublic

Deign86/deped-math-qwen2.5-7b-checkpoint-700-lora

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes12downloads
Model Card

DepEd Math Tutor LoRA (Checkpoint 700)

This repository contains the LoRA adapter checkpoint at training step 700 for a Grade 11-12 mathematics tutoring assistant aligned to Philippine DepEd senior high school topics.

Model Details

Model Description

  • —Developed by: Deign Lazaro (MathPulse AI)
  • —Funded by: Self-funded / independent development
  • —Shared by: Deign86
  • —Model type: PEFT LoRA adapter (not a standalone merged full model)
  • —Language(s): English, Filipino (Taglish instructional style)
  • —License: Apache-2.0 (inherits base model license constraints)
  • —Finetuned from model: unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit

Model Sources

  • —Training workflow: Kaggle notebook pipeline with Unsloth + PEFT QLoRA
  • —Checkpoint source run: deignlazaro/qwen25-deped-lora-production (Kaggle)
  • —Current artifact: Step-700 adapter snapshot selected as the last stable checkpoint before further-loss instability

Uses

Direct Use

This adapter is intended for:

  • —Grade 11-12 math tutoring
  • —Step-by-step worked solutions
  • —Topic explanation and guided practice prompts

Because this is a LoRA adapter, load it on top of the base model.

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit"
adapter_id = "Deign86/deped-math-qwen2.5-7b-checkpoint-700-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

Downstream Use

Recommended downstream use cases:

  • —Classroom support copilots
  • —Practice-question explanation bots
  • —Internal educational QA tools for SHS math review

Out-of-Scope Use

Not intended for:

  • —High-stakes grading decisions without human review
  • —Medical, legal, or safety-critical advice
  • —Formal theorem proving or guaranteed symbolic correctness

Bias, Risks, and Limitations

  • —May produce incorrect arithmetic or overconfident reasoning.
  • —Performance is domain-tilted toward SHS math instruction style and may degrade outside that scope.
  • —Explanations can vary in quality depending on prompt clarity and problem formatting.
  • —Should be used with human oversight for exams, assessment feedback, and official school outputs.

Recommendations

  • —Use deterministic decoding for tutoring outputs where consistency matters.
  • —Add lightweight answer checking (numeric/unit checks) in production.
  • —Keep fallback routing to stronger/general models for out-of-domain prompts.
  • —Run periodic red-team prompts for hallucination and curriculum drift checks.

Training Details

Training Data

  • —DepEd SHS (Grades 11-12) mathematics tutoring-style instruction data
  • —Curriculum focus includes General Mathematics, Statistics and Probability, Business Math, and Precalculus
  • —Data formatted for chat-style supervised fine-tuning

Training Procedure

  • —Method: QLoRA (Unsloth + PEFT)
  • —Base model: 4-bit quantized Qwen2.5-7B Instruct variant
  • —Checkpointing: periodic step checkpoints; this card tracks step-700
  • —Rationale for selection: selected as the last stable checkpoint before loss escalation in continued training
Training Hyperparameters (core)
  • —LoRA rank (`r`): 32
  • —LoRA alpha: 16
  • —LoRA dropout: 0.05
  • —Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
  • —T4-safe sequence length during stable runs: 1024
Compute
  • —Hardware type: NVIDIA Tesla T4 (Kaggle)
  • —Precision regime: 4-bit base model + LoRA adapters

Evaluation

Current Status

This release is a checkpoint artifact for integration testing and iterative development. No formal benchmark table is claimed in this card.

Practical Validation Guidance

Before production promotion, validate on:

  • —Topic coverage by grade-level competencies
  • —Numerical accuracy on held-out problem sets
  • —Explanation quality and rubric alignment
  • —Hallucination/error-rate under ambiguous prompts

Technical Specifications

Architecture and Objective

  • —Architecture family: Qwen2.5 (causal LM)
  • —Adaptation method: PEFT LoRA adapter layers
  • —Objective: instruction-following for educational math tutoring interactions

Files in This Repo

  • —adapter_model.safetensors
  • —adapter_config.json
  • —Tokenizer artifacts (tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, etc.)

Model Card Authors

  • —Deign Lazaro (Deign86)

Model Card Contact

For feedback, use the Hugging Face Discussions tab on this model repository.