John1604/DeepSeek-R1-Distill-Qwen-32B-gguf
3408
Deepseek-r1 32B
This is distilled Deepseek-r1. Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.
Use the model in ollama
First download and install ollama.
https://ollama.com/download
Command
in windows command line, or in terminal in ubuntu, type:
ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-32B-gguf:q3km
(q3km is the model quant type, q5ks, q4km, ..., can also be used)
C:\Users\developer>ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-32B-gguf:q3_k_m
pulling manifest
...
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
Use the model in LM Studio
download and install LM Studio
https://lmstudio.ai/
Discover models
In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.
In the "Mission Control" search bar, type "John1604/DeepSeek-R1-Distill-Qwen-32B-gguf" and check "GGUF", the model should be found.
Download the model.
you may choose the quantized type in the download window.
Load the model.
Load the model to the LM Studio.