Modelpublic

quantflex/SmallThinker-3B-Preview-abliterated-GGUF

sourceHugging Faceotherupdated 2y agoView on Hugging Face

2likes135downloads

Model Card

[image]

GGUF Quants for: huihui-ai/SmallThinker-3B-Preview-abliterated

Model by: huihui-ai (thank you!)

Quants by: quantflex

Run with llama.cpp:

./llama-cli -m SmallThinker-3B-Preview-abliterated-Q5_K_M.gguf -p 'You are a helpful assistant.' --temp 0.7 --top-p 0.8 --top-k 20 --repeat-penalty 1.1 -cnv --chat-template chatml```