CoolFace
Modelpublic

hari31416/deepseek-r1-grug-adapters

sourceHugging Faceapache-2.0updated 24d agoView on Hugging Face
0likes
Model Card

DeepSeek-R1 Grug Reasoning Adapters (7B & 1.5B)

This repository houses all trained fine-tuned LoRA adapter weights across both phases of the Grug Reasoning Project:

  1. 1.DeepSeek-R1-Distill-Qwen-7B: 4-bit QLoRA Supervised Fine-Tuned (SFT) and Direct Preference Optimized (DPO) adapters trained on CUDA (NVIDIA T4).
  2. 2.DeepSeek-R1-Distill-Qwen-1.5B: LoRA adapters trained using the MLX framework on Apple Silicon (M4 GPU).

The goal of Grug reasoning is aligning models to think in compressed, telegraphic fragments inside <think>...</think> tags—cutting conversational filler and generation latency while preserving mathematical task accuracy.

Repository Organization

text
adapters/
├── deepseek-r1-7b/
│   ├── sft/                      # 7B SFT LoRA adapters (PEFT safetensors)
│   └── dpo/                      # 7B DPO LoRA adapters (PEFT safetensors)
├── deepseek-r1-1.5b/
│   ├── it-1/                     # 1.5B Proof of Concept MLX adapters
│   ├── it-2-regularized/         # 1.5B Regularized MLX adapters (prompt-dropout)
│   └── it-2-unregularized/       # 1.5B Unregularized MLX adapters
├── sft/                          # Backward compatible pointer to 7B SFT
└── dpo/                          # Backward compatible pointer to 7B DPO

Benchmark Evaluation Summary

1. DeepSeek-R1-7B (Full GSM8K Test Split, 1,319 Samples)

Model VariantTest SamplesAccuracyFormat ComplianceMean Thinking TokensMean Answer TokensMean Latency
Base Model (7B)1,31975.97%99.85%122.5160.46.09s
SFT Adapter (7B)1,31972.18%94.62%107.7107.36.75s
DPO Adapter (7B)1,31975.44%99.85%122.3162.16.39s
  • —Answer Brevity: SFT cut final answer lengths from 160.4 to 107.3 tokens (33.1% reduction).
  • —Preference Alignment: DPO preference pairs eliminated repetitive derivation loops, restoring format compliance to 99.85% and task accuracy to 75.44%.

2. DeepSeek-R1-1.5B (Apple Silicon Experiments)

ConfigurationAccuracyMean Thinking TokensMean Total TokensMean LatencyFormat Compliance
Base Normal (1.5B)64.9%219.0477.40.88s96.6%
FT Normal (1.5B)66.0%156.2389.30.73s98.9%
FT Regularized (1.5B)54.6%135.0214.70.61s98.2%

How to Use

Loading 7B Adapters with Hugging Face Transformers & PEFT

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
adapter_repo = "hari31416/deepseek-r1-grug-adapters"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

# Load SFT adapter
sft_model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder="deepseek-r1-7b/sft")

# Or load DPO adapter
# dpo_model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder="deepseek-r1-7b/dpo")

Loading 1.5B Adapters with Apple Silicon MLX

bash
# Load and generate using MLX LoRA
mlx_lm.generate \
    --model mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-4bit \
    --adapter-path <path-to-downloaded-1.5b-adapters> \
    --prompt "Janet has 16 eggs..."

Related Resources