Modelpublic

Farhankhan1/deepseek-r1-distill-qwen-1.5b-reasoning-finetune

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face

0likes625downloads

Model Card

DeepSeek-R1-Distill-Qwen-1.5B — Reasoning Fine-tune

—Developed by: Farhankhan1
—License: apache-2.0
—Finetuned from model: unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Training details

—Method: LoRA (r=16, alpha=16), merged into base weights (16-bit)
—Dataset: 1,000 samples from Magpie-Align/Magpie-Reasoning-V2-250K-CoT-Deepseek-R1-Llama-70B
—Steps: 60 (short practice run, not a full epoch)
—Hardware: Google Colab, T4 GPU
—Optimizer: adamw_8bit, lr=2e-4, linear schedule

Prompt template

This model expects the following format — raw/unformatted prompts will degrade output quality:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:
{response}

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Farhankhan1/deepseek-r1-distill-qwen-1.5b-reasoning-finetune")
tokenizer = AutoTokenizer.from_pretrained("Farhankhan1/deepseek-r1-distill-qwen-1.5b-reasoning-finetune")

Notes

This is a learning/experimental fine-tune, not intended for production use. Trained on a small sample for pipeline practice.