Modelpublic

leeminwaan/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes163downloads

Model Card

Model Card for DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF

This repository contains multiple quantized versions of the DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1 model in GGUF format. It is intended for efficient inference on consumer hardware, making large model deployment more accessible.

Model Details

Model Description

—Developed by: leeminwaan
—Funded by [optional]: Independent project
—Shared by [optional]: leeminwaan
—Model type: Decoder-only transformer language model
—Language(s) (NLP): English (primary), multilingual capabilities not benchmarked
—License: Apache-2.0

Model Sources

—Repository: Hugging Face Repo
—Paper [optional]: Not available
—Demo [optional]: To be released

How to Get Started with the Model

python

from huggingface_hub import hf_hub_download

model_path = hf_hub_download("leeminwaan/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF", "DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-q4_k_m.gguf")
print("Downloaded:", model_path)

Quantized versions available:

—Q2\K, Q3\K\S, Q3\K\M, Q3\K\_L
—Q4\0, Q4\1, Q4\K\S, Q4\K\M
—Q5\0, Q5\1, Q5\K\S, Q5\K\M
—Q6\K, Q8\0

Training Details

Training Data

—Based on DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1 pretraining corpus (public large-scale web text, open datasets).
—No additional fine-tuning was performed for this release.

Training Procedure

—Original DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1 → quantized to GGUF formats.

Quantization Results

Quantization	Size (vs. FP16)	Speed	Quality	Recommended For
Q2_K	Smallest	Fastest	Low	Prototyping, minimal RAM/CPU
Q3KS	Very Small	Very Fast	Low-Med	Lightweight devices, testing
Q3KM	Small	Fast	Med	Lightweight, slightly better quality
Q3KL	Small-Med	Fast	Med	Faster inference, fair quality
Q4_0	Medium	Fast	Good	General use, chats, low RAM
Q4_1	Medium	Fast	Good+	Recommended, slightly better quality
Q4KS	Medium	Fast	Good+	Recommended, balanced
Q4KM	Medium	Fast	Good++	Recommended, best Q4 option
Q5_0	Larger	Moderate	Very Good	Chatbots, longer responses
Q5_1	Larger	Moderate	Very Good+	More demanding tasks
Q5KS	Larger	Moderate	Very Good+	Advanced users, better accuracy
Q5KM	Larger	Moderate	Excellent	Demanding tasks, high quality
Q6_K	Large	Slower	Near FP16	Power users, best quantized quality
Q8_0	Largest	Slowest	FP16-like	Maximum quality, high RAM/CPU

Note: - Lower quantization = smaller model, faster inference, but lower output quality. - Q4KM is ideal for most users; Q6K/Q80 offer the highest quality, best for advanced use. - All quantizations are suitable for consumer hardware—select based on your quality/speed needs.

Technical Specifications

Software

—llama.cpp for quantization
—Python 3.10, huggingface\_hub

Citation

BibTeX:

bibtex

@miscDeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF,
  title=DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF Quantized Models},
  author={leeminwaan},
  year={2025},
  howpublished={\url{https://huggingface.co/leeminwaan/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF}}
}

APA:

leeminwaan. (2025). DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF Quantized Models [Computer software]. Hugging Face. https://huggingface.co/leeminwaan/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF

Glossary

—Quantization: Reducing precision of weights to lower memory usage.
—GGUF: Optimized format for llama.cpp inference.

More Information

—This project is experimental.
—Expect further updates and quantization benchmarks.

Model Card Authors

—leeminwaan

Model Card Contact

—Hugging Face: leeminwaan