eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v4

DeepSeek-R1-Distill-Qwen-1.5B Fine-Tuned on GSM8K with Chain-of-Thought Augmentation

Model Overview

This model is a fine-tuned version of DeepSeek-R1-Distill-Qwen-1.5B, trained on the OpenAI GSM8K dataset, augmented with Chain-of-Thought (CoT) reasoning using DeepSeek-V3. The fine-tuning process enhances the model’s mathematical problem-solving abilities, allowing it to provide step-by-step solutions with deeper reasoning.

🔹 Key Features

• —Base Model: DeepSeek-R1-Distill-Qwen-1.5B
• —Fine-Tuned On: GSM8K dataset with DeepSeek-V3-enhanced reasoning
• —Improved Mathematical Reasoning: Generates detailed step-by-step CoT explanations
• —Optimized for GRPO Training: Trained using trl and unsloth for efficient fine-tuning

📊 Dataset & Training Details

• —Dataset: eagle0504/openai-gsm8k-enhanced-using-together-ai-deepseek-train8k-test1k-v1
• —8K train samples, 1K test samples
• —Contains question, answer, and CoT reasoning
• —Training Methodology:
• —Used Guided Reinforcement Policy Optimization (GRPO) via trl
• —Applied gradient accumulation to manage larger batch sizes
• —Integrated DeepSeek-V3 augmentation for enhanced logical reasoning
• —Fine-tuning Tools:
• —Unsloth for memory-efficient Llama-based tuning
• —Hugging Face Transformers for model training

For those interested in replicating the fine-tuning process, I have shared an updated Colab notebook 📓: 🔗 Colab Notebook

You will need: ✅ Hugging Face Token ✅ Together.AI API Key ✅ Unsloth Package

🚀 How to Run the Model (Mac via `llama.cpp`)

Yes! You can run this model locally on macOS using llama.cpp.

1️⃣ Install Homebrew (If Not Installed)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Then add Homebrew to your PATH:

echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"

2️⃣ Install `llama.cpp`

brew install llama.cpp

3️⃣ Run the Model with `llama-cli`

llama-cli -hf eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced-gguf:Q8_0

4️⃣ Alternative: Run Locally via GGUF

mkdir -p ~/llama_models && cd ~/llama_models
wget https://huggingface.co/eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced-gguf/resolve/main/Q8_0.gguf
llama-cli -m ~/llama_models/Q8_0.gguf --interactive

📌 How to Use Model via Python (`transformers`)

You can load the model with Hugging Face Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "A farmer has 24 apples. He gives 6 to each of his 3 children. How many does he have left?"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_length=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))

🔬 Expected Performance

Compared to the base DeepSeek-R1-Distill-Qwen-1.5B, this fine-tuned model:

• —Provides more detailed Chain-of-Thought (CoT) explanations for GSM8K problems.
• —Improves logical reasoning and step-by-step answer formulation.
• —Generates clearer, more structured solutions, making it ideal for educational use.

🗂 Model Hosting & License

📌 Model on Hugging Face Hub: 👉 [eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced](https://huggingface.co/eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced)

📜 License: MIT License – Open for modification and distribution.

If you have feedback or ideas for improvement, feel free to reach out! 🚀🔥

#AI #MachineLearning #DeepSeek #GSM8K #LLM #ChainOfThought #HuggingFace #GRPO #Reasoning