John1604/DeepSeek-R1-Distill-Qwen-14B-gguf

Deepseek-r1 14B

This is distilled Deepseek-r1. Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.

Use the model in ollama

First download and install ollama.

https://ollama.com/download

Command

in windows command line, or in terminal in ubuntu, type:

ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-14B-gguf:q3km

(q3km is the model quant type, q5ks, q4km, ..., can also be used)

C:\Users\developer>ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-14B-gguf:q3_k_m
pulling manifest
...
verifying sha256 digest
writing manifest
success

>>> Send a message (/? for help)

Use the model in LM Studio

download and install LM Studio

https://lmstudio.ai/

Discover models

In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.

In the "Mission Control" search bar, type "John1604/DeepSeek-R1-Distill-Qwen-14B-gguf" and check "GGUF", the model should be found.

Download the model.

Load the model.

Ask questions.

quantized models