Modelpublic

DexterSptizu/qwen2.5-3b-reasoning-lora-bf16

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes

Model Card

Qwen2.5-3B Reasoning (16-bit BF16 LoRA Adapter)

This repository contains the fine-tuned weights for `Qwen/Qwen2.5-3B-Instruct` trained to produce explicit step-by-step <think>...</think> Chain-of-Thought (CoT) reasoning.

Model Summary

—Fine-Tuning Methodology: Method 2: LoRA (Unquantized 16-bit BF16 Base + 16-bit LoRA Adapters)
—Base Model: `Qwen/Qwen2.5-3B-Instruct`
—Trainable Parameters: 18.4 Million (0.59% trainable parameters)
—Dataset: `FreedomIntelligence/medical-o1-reasoning-SFT` (10,000 instructions)
—Hardware Acceleration: 2x NVIDIA H200 NVL GPUs (288 GB VRAM)

Empirical Benchmark Performance

Metric / Feature	Benchmark Score
CoT Reasoning Tag Rate (`<think>`)	`95.0%`
Holdout Benchmark Accuracy	`100.0%`
Training Duration	`11.78 minutes`
Final Epoch Loss	`1.428`

Usage Instructions

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-3B-Instruct"
adapter_model_id = "DexterSptizu/qwen2.5-3b-reasoning-lora-bf16"

# Load base model & tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Attach fine-tuned reasoning adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)

# Inference Example
prompt = "<|im_start|>system\nYou are a helpful reasoning assistant.<|im_end|>\n<|im_start|>user\nSolve step by step: What is 15 * 24?<|im_end|>\n<|im_start|>assistant\n<think>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation & Acknowledgements

—Base model developed by Alibaba Cloud (Qwen Team).
—Fine-tuning executed on dual NVIDIA H200 NVL GPUs.