Modelpublic

KevHamm07/qwen2.5-coder-7b-instruct-hermes-lora-smoke-gguf

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes68downloads

Model Card

qwen2.5-coder-7b-instruct-hermes-lora-smoke-gguf

GGUF conversion of KevHamm07/qwen2.5-coder-7b-instruct-hermes-lora-smoke, a LoRA adapter trained from Qwen/Qwen2.5-Coder-7B-Instruct.

Files

—qwen2.5-coder-7b-instruct-hermes-lora-smoke-q4_k_m.gguf — Q4KM — 4.36 GiB
—qwen2.5-coder-7b-instruct-hermes-lora-smoke-q5_k_m.gguf — Q5KM — 5.07 GiB

Ollama

bash

hf download KevHamm07/qwen2.5-coder-7b-instruct-hermes-lora-smoke-gguf qwen2.5-coder-7b-instruct-hermes-lora-smoke-q4_k_m.gguf --local-dir ./hermes-qwen-gguf
cat > ./hermes-qwen-gguf/Modelfile <<'EOF'
FROM ./qwen2.5-coder-7b-instruct-hermes-lora-smoke-q4_k_m.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
{{ end }}"""
PARAMETER stop <|im_end|>
PARAMETER temperature 0.2
PARAMETER num_ctx 4096
EOF
ollama create hermes-qwen-coder-smoke:7b -f ./hermes-qwen-gguf/Modelfile
ollama run hermes-qwen-coder-smoke:7b