Modelpublic

abhinav241998/qwen3-4b-fitsense-qlora

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes28downloads

Model Card

FitSenseAI — Qwen3-4B QLoRA Adapter

A LoRA adapter fine-tuned on top of unsloth/Qwen3-4B for AI-powered fitness coaching. The model is a tool-calling agent that generates personalized workout plans, logs workouts, tracks health metrics, analyzes progress, and answers fitness coaching questions.

Available Formats

Format	Path	Description
LoRA adapter	`/` (repo root)	QLoRA adapter weights, load with PEFT
BF16 merged	`final_merged/bf16/`	Full merged model in bfloat16
AWQ quantized	`final_merged/awq/`	4-bit AWQ quantized merged model

Model Details

—Base model: unsloth/Qwen3-4B
—Fine-tuning method: QLoRA (4-bit base + LoRA adapters)
—Task: Supervised fine-tuning (SFT) on synthetic fitness coaching conversations
—LoRA rank: 8 | LoRA alpha: 16 | Dropout: 0
—Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
—Training steps: 270 over 3 epochs
—Training time: 42m 36s
—Max sequence length: 16,500 tokens
—Precision: bfloat16

Training Hyperparameters

Parameter	Value
Learning rate	3.46e-4
LR scheduler	cosine
Warmup ratio	0.05
Batch size	1
Gradient accumulation steps	8 (effective batch 8)
Max grad norm	1.0

How to Use

LoRA Adapter

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3-4B")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen3-4B")
model = PeftModel.from_pretrained(base_model, "abhinav241998/qwen3-4b-fitsense-qlora")

BF16 Merged Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "abhinav241998/qwen3-4b-fitsense-qlora",
    subfolder="final_merged/bf16",
    torch_dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained(
    "abhinav241998/qwen3-4b-fitsense-qlora",
    subfolder="final_merged/bf16",
)

AWQ Quantized Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "abhinav241998/qwen3-4b-fitsense-qlora",
    subfolder="final_merged/awq",
)
tokenizer = AutoTokenizer.from_pretrained(
    "abhinav241998/qwen3-4b-fitsense-qlora",
    subfolder="final_merged/awq",
)

Framework Versions

—PEFT 0.18.1
—Transformers 4.43+
—TRL 0.9+
—Unsloth