Modelpublic

Renugadevi82/cisco-nx-ai-gguf-q4_k_m

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes14downloads

Model Card

Cisco NX-AI GGUF Q4KM

4-bit quantized (smallest, ~600MB)

Download

bash

wget https://huggingface.co/Renugadevi82/cisco-nx-ai-gguf-q4_k_m/resolve/main/cisco-nx-ai-q4_k_m.gguf

Usage with llama.cpp

bash

./llama-cli -m cisco-nx-ai-q4_k_m.gguf -p "Configure VLAN 100" -n 100

Usage with llama-cpp-python

python

from llama_cpp import Llama
llm = Llama(model_path="cisco-nx-ai-q4_k_m.gguf")
output = llm("Configure VLAN 100", max_tokens=100)
print(output['choices'][0]['text'])

File size: 0.62 GB